Computers & Education 55 (2010) 513e526

Contents lists available at ScienceDirect

Computers & Education

journal homepage: www.elsevier .com/locate/compedu

A conceptual framework mapping the application of information searchstrategies to well and ill-structured problem solving

Kumar LaxmanCentre for Educational Development, Nanyang Technological University, Singapore

a r t i c l e i n f o

Article history:Received 15 September 2009Received in revised form19 January 2010Accepted 12 February 2010

Keywords:Problem-based learningProblem solvingInformation search skills

E-mail address: haribol.kumar@gmail.com

0360-1315/$ e see front matter � 2010 Elsevier Ltd.doi:10.1016/j.compedu.2010.02.014

a b s t r a c t

Problem-based learning (PBL) is an instructional approach that is organized around the investigation andresolution of problems. Problems are neither uniform nor similar. Jonassen (1997, 2000) in his designtheory of problem solving has categorized problems into two broad types e well-structured andill-structured. He has also described a host of mediating skills that impact problem solving outcomes. Thethrust of this paper rests in the argument that the basis of these mediating skills is information searchliteracy and particularly, in view of the utility of the Internet as an informational repository, effectiveinformation searching skills. This study was an investigation of how different Internet informationseeking strategies can be used to engage in problem solving. A conceptual framework that explains howdifferent Internet information searching strategies can be employed in successfully solving well andill-structured problems was devised and empirically tested. The research site was a newly establishedpolytechnic in Singapore that employs problem-based learning to support its curricular implementation.The sample population of students came from a class of 25 first-year students. The research findings ofthis study inform that information searching skills indeed play an important role in problem solving.The findings affirm the need for students to be systematically instructed in the skills of informationsearching to be able to accomplish problem solving. The information searching necessary for solvingwell-structured problems is constrained and readily manageable. Thus, students only have to beacquainted with fundamental information searching skills to solve well-structured problems. On theother hand, the information needs of ill-structured problems are usually complex, multi-disciplinary andexpansive. Hence, students have to be trained to apply a more advanced set of information searchingskills in resolving ill-structured problems.

� 2010 Elsevier Ltd. All rights reserved.

1. Introduction

Solving real-world problems is an effective learning activity that promotes meaningful learning in formal educational settings.Since problem solving forms the backbone of the bulk of activities taking place in the non-school world, creating instructionalenvironments that simulate real-world working contexts enables learning to be meaningful. David Jonassen (1997, 2000) in his designtheory articulates a typology of different kinds of problems that range upon a diverse continuum spanning from well-structured to ill-structured.

Individual differences in various skills-based competencies mediate and affect the modalities of problem solving. In light of the wide-spread reliance upon the Internet as an important information repository, effective information searching and use becomes critical tosuccessfully accomplishing problem solving (Jakes, Pennington, & Knodle, 2002). However, finding relevant information in the expansivelandscape of the Internet is not an easy task as it requires a variety of skills such as understanding the different Internet tools, knowledge ofsearch techniques, cognitive capacity to organize searches, and ability to efficiently execute the searches. These skills are sophisticated andcannot be easily learnt by students on their own. Without being subjected to formal training in information searching literacy, the possi-bilities of students performing haphazard, disorganized searching and appropriation of unreliable information are high. In Singapore,students' Internet information searching skills are usually self-taught. At present, there are no formal instructional curricular programmes inSingapore educational institutions that are designed towards systematically teaching students practical information searching

All rights reserved.

K. Laxman / Computers & Education 55 (2010) 513e526514

competencies. This could potentially lull students into a false sense of complacency about their actual Internet navigational abilities and beignorant of the difficulties involved in locating relevant information.

2. Review of the literature

2.1. Problem solving

Problem solving can be defined as the goal-directed sequence of cognitive operations in finding the unknown to resolve a problemsituation (Anderson, 1980). Problem-based learning (PBL) is an instructional methodology that centrally focuses upon problem solvingactivities. Savery (2006) describes PBL as an instructional (and curricular) learner-centered approach that empowers learners to conductresearch, integrate theory and practice and apply knowledge to solve problems.

2.2. Design theory of problem solving

According to the design theory of problem solving of Jonassen (2000), the ability of a learner to solve problems is a function of the natureof the problem, the way that the problem is represented to the solver, and a host of individual differences in skills possession mediating theprocess. The theory attempts to link the acquisition of prerequisite skills to specific models of problem solving instructions based upona range of problem solving outcomes. Problems can be differentiated by their attributes and goals and this requires problem solvers todiscriminate in the use of appropriate skills and dispositions to engage in problem solving. Generally, problems can vary in terms ofstructuredness. Problems could generally be categorized as being either ill-defined, non-routine and ill-structured or well-defined, routineand well-structured (Jonassen, 1997; Mayer & Wittrock, 1996).

Well-structured problems consist of a well-defined initial state (what is known), a known goal state (nature of the solutionwell-defined)and a constrained set of logical operators (known procedure for solving) (Jonassen, 2000). Well-structured problems also have knowable,comprehensible solutions where the relationship between decision choices and all problem states is known or probabilistic (Wood, 1983).On the other hand, ill-structured problems are the kinds of open-ended, emergent problems that are faced in daily, professional lives. Theirsolutions are neither predictable nor convergent. Not limited by the strict boundaries of content domains, these problems tend to be multi-disciplinary in character (Jonassen, 2000).

2.3. Information searching in problem solving

Information literacy of students fundamentally encompasses the competency levels of students' information searching capabilitiesand the efficacy of information seeking strategies they choose to implement (Jones, 1998). Effective location and use of information tomould robust solutions is essential to problem solving and decision-making (Burchinall, 1976; Zurkowski, 1974). Schaster, Chung andDorr (1998) found in their studies that most students lack these important information searching skills and rarely employ systematicsearch strategies or select appropriate search terms. They also observed that students generally spend little or no time planning theirsearching.

The Internet has emerged as the most powerful digital information medium to shape and define the educational field. This is due to theplethora of educational and information resources that it offers for teachers and students. These resources can be accessed flexibly and ondemand. The Internet also facilitates more autonomy in learning and gives learners control over their own learning progress (Landow,1992).

2.4. Information search strategies

The term search strategy adapted from Bates (1979) can be defined as consisting of a series of concrete actions (steps) aimed at findinginformation. Nachiamas and Gilad (2002) have explicated a taxonomy of three different information search strategies: search enginestrategies, browsing strategies and direct access strategy. Search engine strategies consist of six different types of search engine strategies:Keyword search, wide search definition, complex search, use of general knowledge, computer convention and Boolean search.

Browsing strategies consist of two techniques. The first one is what Nachiamas and Gilad (2002) called as ‘using a directory’whichmeansbrowsing through a directory or catalogue. Such an approach can be found in Yahoo! Website. The other option is to access a specific portalthat is related to the subject of interest. Direct access strategy is direct typing where participants simply type the address of the known URLto access information of the topic to be explored (Nachiamas and Gilad, 2002).

2.5. Inquiry-based learning

Inquiry-based learning is a process where students formulate investigative questions, obtain factual information and then buildknowledge that reflects their answer to the original question (Jakes et al., 2002). Inquiry-based learning promotes learning experiencesenabling the exploration of theoretical ideas and conceptual change (Teixeira-Dias, De Jesus, de Souza, & Watts, 2005). An inquiry-basedinstructional approach is an integral aspect of problem solving.

3. Research methods and procedures

3.1. Research design

One of the main goals of this study was to formulate and empirically test the efficacy of an interventionist conceptual model that mapsthe application of different information seeking techniques to successfully resolvingwell and ill-structured problem types. It was hoped that

K. Laxman / Computers & Education 55 (2010) 513e526 515

a set of heuristics could be established in linking the efficacy of different Internet information searching strategies to the nature of struc-turedness of multi-disciplinary authentic problems.

This study adopted a within-subjects experimental design as the most pragmatic approach to implementing and validating theproposed framework. This design was chosen since it had the practical advantage of requiring fewer subjects and enables themeasurement of the impact of variations in experimental conditions on learning performance. Qualitative data was collected andanalysed to understand the direct experiences of students in their application of information search skills during their engagement withproblem solving.

3.2. Participants

Participants in this study were a class of twenty-five students from a polytechnic tertiary institution in Singapore that hasimplemented a problem-based learning methodology for all curriculum subjects and at all academic levels of study. In this problem-based learning model, students work in teams of five on a given problem each day of the week for the different disciplinary subjectmodules.

These first-year first-term students were new to the pedagogical approach of problem-based learning since their prior formal schoolingexperiences were traditional and teacher-centered in orientation. Six problems from the curriculum of the ‘cognitive processes and problemsolving’modulewere selectively chosene three being well-defined problemswith the other three being divergent problems. Students wererandomly assigned to their groups and this group assignment was maintained throughout the duration of this study to ensure consistencyand validity in analyzing differences in problem solving performance across the six problems.

3.3. Data collection and analysis procedures

The data collection for this study was done in phases, as outlined in the diagrammatic representation Fig. 1. In phase 1 e the pre-intervention period, an open-ended survey questionnaire was administered to examine the Internet information search literacy practicesand proficiencies of students. In Phases 2, 3, 5, 6, 8 and 9 e the problem solving sessions for the six chosenwell and ill-structured problems,students had to demonstrate their learning achievement for the day by presenting and submitting their end-product artifacts representingtheir solutions to the problem. Students also had to submit their cogitations in their reflection journals for each of the six problem solvingsessions. Phases 4 and 7 e the intervention phases were sandwiched in-between the problem solving sessions so as to be able to comparethe treatment effects of the intervention on students' overall problem solving performances. The pedagogical impact of the informationsearching skills training delivered during phases 4 and 7 was assessed by examining improvements or changes in students' problem solvingcharacteristics, abilities and outcomes in relation to the structuredness of the problems. The data collected for these nine key phases wascontent analysed to draw comparison and insights can be made on students' problem solving performances and information searchingbehaviours for the six problems.

3.4. Intervention

A training intervention was done to determine the treatment effects of different Internet information searching skills and strategies onstudents' abilities to solve different types of problems. The empirical evaluation of the efficacy of this intervention was based upon theanalysis of the impact of these information searching skills on outcomes of problem solving. The framework serves as a flexible structure ofreference and consists of a suite of recursive Internet information searching strategies that equip students with the necessary informationliteracy skills to enhance their problem solving capabilities. The design of this framework as shown in Fig. 2 was primarily based upon aninquiry-based learning approach.

Well-structured problem 1

Pre-intervention phase

Phase 1 Phase 2

Ill-structured problem 2

Phase 3 Phase 4

Intervention skills 1 and 2

Well-structured problem 3

Phase 5

Ill-structured problem 4

Phase 6 Phase 7

Intervention skills 3 and 4

Well-structured problem 5

Phase 8

Ill-structured problem 6

Phase 9

Fig. 1. Schematic layout of data collection sequence.

Skill 1 – Learning about Internet Search Tools

Skill 2 – Using Information Search Strategies and Techniques

Skill 3 – Framing Essential and Foundation Research Questions

Skill 4 – Developing and Organizing Search Keywords

Fig. 2. Training interventionist framework schema.

K. Laxman / Computers & Education 55 (2010) 513e526516

3.5. Skill 1 e learning about Internet search tools

The first skill articulated in the training framework required students to learn about the various types of information search tools foundin the Internet. Students were informed on the working attributes and operational mechanisms of search engines, search directories andmetasearch engines. Students were then encouraged to try using recommended search engines, metasearch engines and search directoriesto evaluate their information searching capabilities and become conversant with the functional purposes of these search tools to be able toutilize them effectively.

3.6. Skill 2 e using information search strategies and techniques

Having introduced students to the search tools offered by the Internet, they were then oriented to Internet information search strategiesand techniques that facilitate efficient information seeking.

One useful search strategy suggested to students was that of using multiple search engines, including metasearch engines during theirinformation searching activities. Students were found to be inclined to access a single search engine for all their information needs. Sincesearch engines usually function in distinct ways and present search results that vary for the same query entered, students were cautionedagainst reliance on a single search engine. Instead theywere encouraged to seekmultiple search engines and their advanced search featuresto carry out their information search.

Students were then familiarized with the next set of information search strategies fromNachiamas and Gilad's (2002) taxonomye directkeyword search, wide search definition, complex search strategy, use of general knowledge and computer conventions involving file suffixessuch as .gif and .jpg.

Having instructed students on the essentials of search strategies and encouraging them to explore these strategies, they were nextintroduced to useful search techniques that complement search strategies.

Some key search techniques that students were acquainted with included the following: Searching for an exact phrase which involvedspecifying the search phrase surrounded by double quotes to find an exact phrase in the document results listed by a search engine,searching for required, excluded and multiple words which involved the use of Boolean searching operators such as pluses, minuses, AND, OR,NOT, being specific in defining keyword search terms and being exactin framing keywords and being distinct by being direct and focused inexecuting searches and finally being concise in their choice of keywords and eliminating unnecessary words. Examples were provided for eachof these techniques to familiarize students on their utility.

3.7. Skill 3 e framing essential and foundation research questions

Jakes et al. (2002) describe essential questions as those that require students to make viable decisions or plan courses of action basedupon sound reasoning. Essential questions are key questions that define the starting point of a search attempt in order to frame the searchprocess. Formulation of student-authored essential questions is a vital component of structured inquiry and marks a departure fromtraditional classroom modes of interactions predominated by teacher talk. Students were presented with numerous examples of essentialquestions to familiarize them with the process of essential questions crafting.

After understanding how essential questions are formulated, crafting of foundation questions was then taught to students. McKenzie(1996) alternatively labels foundation questions as subsidiary questions. Foundation or subsidiary questions, as the names suggest,provide students with the scaffolding structures inworking towards and achieving the search objectives conveyed by the essential question.The answers to foundation questions generate substantive information which then can be processed and integrated to build up coherentsolutions in response to the essential question (Jakes et al., 2002). Students were presented with varied worked examples to be acquaintedwith good questioning skills and then given a few practice problems for them to apply hands-on what they had learnt. Overall, thecombination of worked examples and practice problems within an instructional framework enables the activation of learners' priorknowledge and facilitates the acquisition of a broad range of problem schemas (Jonassen, 1997).

What are the two books written by

Laurence Yep?

Laurence Yep

books

AUTHORSTHEME

FOUNDATION QUESTION

TOPIC

FOCUS

How many blue whales are left in the world?

blue whales

population

WHALES

Fig. 3. One example of keyword-category concept map.

K. Laxman / Computers & Education 55 (2010) 513e526 517

3.8. Skill 4 e developing and organizing search keywords

After students had learnt to model the questioning process, the next and final stage in the intervention framework was the training ofstudents to initiate an action plan to seek answers to the brainstormed essential and foundation questions. This fourth stage of theinterventionwas designed to provide intensive instructions and guided practice to students in training them to efficiently develop keyword-category concept maps after carefully selecting a pool of relevant keywords.

The objective of this stage of the intervention was to educate students on the techniques of keyword-category concept mapping.Eagleton, Guinee and Langlais (2003) have cognitively modeled the process of constructing key-category maps by examining a subject topicin terms of component items such as theme, topic, focus and questions. The theme refers to the broad agenda of the information explorationto be performed and is drawn from the objectives of the essential question. Topic and focus areas are the specific domains of search interestand can directly be linked to each foundation question. The topic and focus inputs then become the keywords for information searching tobe accomplished. Pre-prepared examples of keyword-category concept maps such as the one in Fig. 3 were then shown to studentsto demonstrate how keyword-category concept mapping can be done. Students were then engaged in guided practicewhere they continuedtowork on the practice problems given in phase 3 of the intervention to hone their skills of efficiently developing keyword-category conceptmaps.

3.9. Instruments

To determine the self-perceived levels of Internet information search literacy skills amongst participant students, a pre-interventionsurvey consisting of six open-ended questions was administered to students (a copy of the survey is attached in Appendix A). The surveyinformed the design of the elements of the interventionist framework aimed at raising students' information searching competencies.

3.10. Tasks

Phase 1 well-structured problem was a constrained, well-defined problem. In Jonassen's (2000) typology of problems, this problemwould be classified as a story problem. Situated within a brief story, this arithmetic problem tests students' mathematical and scientificconceptual understanding. Students had to make reasonable assumptions based upon the limited number of parameters embedded withinthis problem. The problem is well-structured since it requires the contextual application of a finite number of regular concepts, rules andaxioms to be able to solve it in a mechanistic and sequential manner.

The PackageThere is an aeroplane that is flying approximately 2e3 km above you. As you look up, you see a package falling out of the aircraft.How far do you have to walk from where you are to retrieve the package?

Phase 2 ill-structured problem was a complex problem that required students to think creatively. This design problem can be charac-terized as ill-structured since there are no easy or obvious textbook solutions that could be devised and the problem inherently hasambiguous specifications in relation to goals, configurations and outcomes. It possesses no pre-defined or fixed solution routes and callsupon the integration of multiple domains of knowledge expertise.

Let's design.....Space is very limited in Singapore and every square meter is precious. In order to maximize space, design a 100 storeys skyscraper thatdoes not use lifts, staircases, or escalators.

Phase 3 well-structured problem entitled ‘Look before you jump’ was the first well-structured problem that was given to students tosolve after they had undergone the first and second stages (basic tier) of the intervention training programme. This problemwas classified asa well-structured one since the problem solving processes, strategies and outcomes were bounded and close-ended in character with

K. Laxman / Computers & Education 55 (2010) 513e526518

convergence in solutions highly likely. In order to solve this problem, students had to formulate effective methods of comparing figures inmathematically acceptable ways to ascertain which of the two situations is significantly more critical.

Look before you jump.Consider the following situation:In the year 2005, Thailand had 34,291 cases of dengue cases recorded.In that same year, Indonesia experienced about one and a half times the number of cases. You have a smaller chance of contractingdengue in Thailand than in Indonesia.Examine the reasoning behind this conclusion. Evaluate potential pitfalls of making conclusions with limited information. How wouldyou respond in this situation?

Phase 4 ill-structured problem entitled ‘What do you mean’was an ill-structured problem that was given after participant students hadcompleted training of the basic tier of the interventionist training programme. Being a situated ill-defined problem, the problem elementswere not clearly spelt out, the goals or outcomes of the problemwere notwell articulated, the problem containedmultiple inquiry paths andinformation needed to solve the problem was not explicitly stated in the problem statement. Typically, this is a social dilemma problem inJonassen's typology of problems.

What Do You Mean?Examine the images on the next few slides. What makes up a concept? What is meant by fat?

K. Laxman / Computers & Education 55 (2010) 513e526 519

K. Laxman / Computers & Education 55 (2010) 513e526520

Phase 5 problem was dealt with by students after having attended both phases of the interventionist framework. This problem can becharacterized as well-structured and bounded in scope. In Jonassen's continuum of problem typologies, this problem could be classified asa blend between algorithmic and story based problems. The mathematical ways in which this problem can be approached and decon-structed are limited. A constrained set of elements encompass the problem space and they have to be arithmetically manipulated in seekingsolutions. The computed solution for this problem tends to be convergent and predictable.

Who is it?An incomplete set of bones was recovered from an abandoned warehouse in China. Investigators strongly suspect that the bones belongto a Chinese male.As part of the Forensic Anthropologist teamworking to recreate a physical impression of the victim, you are given the femur and asked togauge the victim's height from it.

More information is given below:

K. Laxman / Computers & Education 55 (2010) 513e526 521

� Bone Stats and description� Length e 55 cm� Thickness e 7.5 cm� Age e Estimated 17e21 years� Findings e Bone knobs are grooved and show signs of wear, indicating that the owner might have been physically active. Scarring

along the middle portion of the bone (not shown), strongly suggests a hairline fracture sustained in the past. Age of fracture isestimated to be within the last 8 years.

The sixth and final problemwas given to students after both tiers of the training program had been completed. This was primarily an ill-structured problem since it was designed to yield larger individual variability in problem solving strategies and solution outputs. What givesthis problem its ill-structuredness are the multiple solution pathway options solvers face in identifying the essence of the problem, devisinggoals, formulating possible solutions and choosing the best solution.

Candle in the WindAnalyse a candle and the various representative meanings associated with the candle to construct better understandings of thesignificance of a candle.

4. Findings & discussion

4.1. Pre-intervention survey

Analysis of students' responses to the items in the pre-intervention survey highlighted their lack of fluency in information search skillsand the pertinent need of the interventionist training to upgrade their skill-levels. Twenty-two out of the twenty-five students mentionedthat all their attempts at learning to use the Internet was mainly through self-taught efforts, random trial and error or help from novicefellow students or friends. Only three students indicated that they learnt to use the Internet through intentional instruction provided bytheir primary/secondary schools. The student feedback to this first question provided evidence that there is a lack of formal learning ofinformation search literacy in schools and tertiary institutions. This issue is of significant relevance since training students to be informationliterate in formal school settings assumes importance in light of a working world that is increasingly becoming digital informationorientated, with prevalent dependence upon the Internet for information to inform decision-making.

In response to the second question in the questionnaire, twelve out of the twenty-five students replied that they do not really plan fora search and normally plunged headlong into searching for information. Some reasons ascribed by these students to such a tendency werelack of prior knowledge, skills and instructional training in understanding how information searches ought to be planned, the perceivedwaste of time and no recognition of any explicit need for modeling information planning and management. Many of the reasons raised bythese students reveal their misconceptions or ignorance of the utility of information search planning and organization in carrying out theirsearch activities.

The third question queried on students' knowledge of search tools and the reasons behind their choice of search tools in carrying outtheir information searching. Almost all the surveyed students expectedly mentioned search engines as the common search tools theyfrequently access. The fourth question in the survey probed students' patterns of preferences in using search engines and their supportingreasons. Not surprisingly, amajority of the students i.e. twelve studentsmentioned Yahoo! and another eight students stated Google. Yahoo!and Google were the two most popular choices of search engines amongst Internet users of all ages. Three students also specified MSN, AskJeeves, Lycos and Ask.com. Some of the reasons attributed by students to their preference for Yahoo! and Google include thewide popularityof these search engines, unfamiliarity with other search engines, the perceived relevance of the search results listed and the convenience ofthe features provided including offering of tool bars and presentation of brief information statements on each search result.

The fifth questionwas an extension to the fourth. The objective of this questionwas to investigate students' familiarity with the plethoraof available search engines and their awareness of which of these search engines best suited their contextual needs. Vansickle (2000) arguedthat using multiple search engines allows for more extensive searching of the Web since even top ranked search engines have a relativelylow degree of coverage overlap with one another. Significantly, twelve students responded that they use only one search engine to carry outtheir information searches. These students did not realize that one single search engine would not fully satisfy in entirety all their infor-mation needs. Generally, accessing multiple search engines would be a more advisable technique since the information output would bemore comprehensive, with a greater degree of success in finding relevant, cross-referenced information. The remaining thirteen studentsgave the feedback that they usually use multiple search engines in performing their Internet information searching. Though, evidently therewas a lack of depth in the explanations provided by the students on the underpinning reasons for their use of multiple search engines, atleast, at a fundamental level, they were able to recognize the benefits of leveraging upon multiple search engines.

The sixth question was framed in alignment with the objective of investigating students' awareness of search engines and the structuralsimilarities/differences of the search methodologies employed by these search engines. Nine students provided the honest feedback that

K. Laxman / Computers & Education 55 (2010) 513e526522

they did not know or were not sure if there existed any differences/similarities in the ways in which the variety of search engines operate,with one student commenting that he never really felt it was a critical issue to want to find out more. Six students were of the erroneousopinion that there are no differences between the different search engines in terms of their functional attributes. They had the miscon-ception that all search engines search for information in the same manner and the information displayed upon initiating a search requestwould also be similar. Students' feedback to this question reinforced the dominant finding that participant students obviously lacked robustInternet information search literacy acumen.

4.2. Phase 1 problem (well-structured)

Teams 1, 3 and 4made the right assumptions and successfully solved this givenwell-structured problem. This they did by first calculatingthe unknown parameter of time taken for the package to land on the ground by using the kinematics formulae s ¼ ut þ 1/2at2. Substitutingthe rounded up figure of 10m/s2 for acceleration due to gravity and rearranging the variables in the formulae, the time taken for the packageto dropwas easily found. Applying this known time factor students in these three groups used the formulae distance¼ speed� time to derivethe desired answer for the distance to be walked to recover the package. Team 2 only managed to partially solve the problemwhile Team 5was unsuccessful in solving the problems since either they made the wrong assumptions or conceptually applied incorrect formulae. Theseresults support the argument that well-structured problems are more easily solved through prescriptive ways by using an organized set oflogical operators and procedures. Students recognized the problem statement to be a physics and/or mathematics related algorithmicproblem when the problem statement was first issued to them.

The learning difficulties experienced by students during this particular problem solving session were attributed to their inchoateinformation searching skills. Students noted in their ruminations in their reflection journals in response to the question on how they wentabout searching for information to solve this problem that the major obstacle they faced was in trying to locate the right resources in theInternet to inform them on the problem solving process. Students were unconfident of the most effective means of organizing theirinformation searching to optimize their information activities. Most students preferred to constrain their searches to using a single searchengine such as Yahoo! without realizing the usefulness of the multitude of other search engines. Hence, students were not aware of thestrengths and shortcomings of the various search engine tools that could have been judiciously utilized to improve search quality. Manystudents also had entered lengthy search phrases and at times, even sentences as search terms. This contributed to diffused search outputand resultant frustration. The lack of rigour in students' information searching skills in being able to devise coherent search action plans,craft precise search terms, locate and apply relevant information was evident in analyzing students' reflection journal inputs.

4.3. Phase 2 problem (ill-structured)

This being an ill-structured, open-ended design problem, most students were baffled at brainstorming for ideas and faced numerousdifficulties during the problem solving process. This is hardly surprising considering the constraints imposed by the problem e studentswere faced with a paradoxical problem scenario where the proposed design model had to be futuristic as well realistic. Team one suggestedtravelators as their preferred choice of transportation means in servicing the 100 storeys of the skyscraper. Team two came up with thedesign prototype of a pulley system. Team three chose a building design that focused on a ballast ‘lift’ concept. Team four chose a spiral linkconcept as their design solution. Team five in their presentation proposed a design blueprint that was a combination of the architecturalideas and structural features of the Taipei 101 and Burj towers.

In view of the stipulated constraints embedded within the problem trigger, normative solutions would not be adequate in satisfying therequirements of the problem. Thus, students were forced to think divergently and creatively. Not surprisingly, in the end, only two of the fivegroups were able to produce design models that were acceptable, resourceful and implementable. Teams one, two and five were unsuc-cessful in their attempts at solving the problem. Team three performed credibly well and team four produced the most impressive andsophisticated design solution for this problem solving task. This shows that students find ill-structured problems much more difficult tosolve than well-structured ones such as the earlier problem 1 of this study.

In their reflection journal postings on how they went about their information searching to solve this problem, students noted that theyfound the scope of the information search for the given problem to bewide in ambit and challenging. Being an ill-structured design problem,it was not possible to agree upon a definitive solution and thus, there were no standardized right or wrong answers. Thus, it was hardlysurprisingwhen students prevalently commented that theywere not sure even on how they should commence their information search andwhat appropriate search terms they ought to enter to execute search actions. Compounding the problem further, students had received littleor no formal instructions on competency-based information literacy skills during their pre-polytechnic schooling days. This meant thatstudents were not adequately trained in the rigours of these skills and thus, hampered in their attempts at processing the information needsof the multi-faceted design problem from various perspectives. Enclosed are some sample excerpts from students' inputs that amplydemonstrate students' acute lack of effective information searching abilities and the resultant difficulties they encountered:

“We first searched the Internet on the differences between escalator and travelator. Whenwe confirmed them to be different, we decidedto put in our solution. We tried to search for building construction sites but couldn't find any relevant information. Thus we had to solvethe problem by relying on prior knowledge and real-life examples we have read up before.” (student L)“The problem question was written such that it is not possible to copy an existing transportation system elsewhere in the world. Therewere lots of factors to be concerned about e.g., safety of passengers etc. We search hard on the Internet for information by looking fordifferent types of transportation. Some of our teammembers tried to even do a search under ‘future transportation.’ But it all didn't helpus much.” (student M)

4.4. Phase 3 problem (well-structured)

Impressively, all five teamsmanaged to correctly solve this given problem. Taking into account the populations of the countries, studentsdid their calculations by deriving the incidence rate values using the following formulae:

K. Laxman / Computers & Education 55 (2010) 513e526 523

Incidence rate ¼ No: of casesTotal population

� 100%

For 2005, Indonesia's incidence rate of 0.026% was found to be lower than Thailand's incidence rate of 0.053%.Students in their reflection journal inputs for this problem were prompted to articulate their feedback on how useful they found the

application of the information search skills they had been taught in solving the given problem. All students agreed that the deliveredinformation literacy training on skills 1 and 2 positively improved their information seeking abilities and well-structured problem solvingcognition. Being aware of the different information searching techniques enabled students to achieve better problem solving mastery.Students explained that the repertoire of skills learnt expanded their information searching capabilities and enhanced the quality of theirproblem solving experiences. This resulted in better learning engagement, productivity and outcomes. This was amply demonstrated bystudents' enhanced problem solving performance in successfully tackling the given well-structured problem after having undergone thefirst phase of the training interventionist program.

Overall, students strongly attributed their improved learning performance for this well-structured problem to the instructional efficacyof the interventionist training programme. Some excerpts from students' contemplations supporting these findings are attached herein forreference:

“I now know how important Internet information searching skills are in problem-based learning environments. And the skills taught tous previously helped us tremendously in solving today's problem.” (student B)‘Before the training I did not know there were so many various search strategies such as Boolean operators. I used them today and easilyfound the relevant information to solve today's problem.” (student J)

4.5. Phase 4 problem (ill-structured)

Team 1 analysed the different positive and negative perceptions of fats.Team 1 was of the opinion that there is no definitive way of defining fatness. It is context dependent and contoured by one's value

system. Team 2 attempted to define fats and then explained the advantages of fats. The team viewed a concept such as fats as somethingthat affects perceptions and behaviours. Team 3 explained that factors such as culture, environment and society impact humanperceptions of a concept. Students in this team noted the general tendency of most people to perceive a fat person as being overweightand not good looking. In summarizing their arguments on this problem, team 3 concluded that a concept is something rich in meaningsand one has to go deeper in probing and exploring a concept. In the instance of fatness, though the majority might view it as beingplump, there are circumstances when being fat is desirable. Team 4 analysed what fats are from a scientific standpoint. Students in thisteam averred that fatness as a concept can be used in a multitude of contexts, depending upon influencing situations or conditions. Team5 included in their presentations the Encarta definition of concept as something that somebody has thought of, or that somebody mightbe able to imagine. Using this definition as the basis, students described someone who is fat to be having a big body size. Students wereof the opinion that we only see one view whereas the dictionary presents different definitions of a word. Overall, teams 1, 3 and 4 wereable to develop and submit satisfactory solutions to the problem. Teams 2 and 5 were not too successful in their attempts at solving thisproblem. The teams that performed credibly well in this problem solving exercise analysed the concept of fatness from different angles ofanalytical reasoning by critically examining each of the given images for their different meanings. The teams that did not fare well failedto probe deeper into the meanings embedded within the notion of fatness and either glossed over or ignored some of the images insubstantiating their viewpoints.

When instructed to post their views in their reflection journals on the utility of the taught information search skills in solving this ill-structured problem, similar to the earlier well-structured problem, students registered significantly positive feedback on the usefulness ofthese skills. Students had become better equipped in filtering the essentials from the non-essentials in locating relevant web information tosolve the given problem. Student H aptly remarked that “..this problem is open-ended and there are lots of supporting information to besearched for amongst hundreds and thousands of websites. So the basic techniques taught were useful in getting specific informationwithina short time. I applied extensively these skills today in searching for information and better understanding concepts such as fats.” Studentsmentioned that they had successfully applied techniques such as accessing multiple search engines and using Boolean operators. There isevidently a positive correlation between these skills and ill-structured problem solving competencies. However, as evidenced by themoderate success rates in solving the problem and the textual analysis of themusings of students in their reflection journals, students foundthe taught basic information searching skills in themselves to be inadequate in rigour in fully tackling the complexities embeddedwithin ill-structured problems. These fundamental skills only partially fulfilled their information requirements in being able to comprehensivelylocate needed information. There was a need for more instructional mediation in further raising students’ information searching profi-ciencies by being acquainted with more robust information searching strategies. For example, student Q commented that “I generally foundapplying the information search skills taught to us to be helpful in solving the problem but it wasn't enough. The problemwas difficult andwe struggled. I think we need more help in developing our information skills to tackle these kinds of problems.” These concerns wereappropriately addressed during the second phase of the interventionist training involving the advanced tier of information search skills.

4.6. Phase 5 problem (well-structured)

All five teams were successful in generating valid solutions for the problem and accomplishing targeted learning outcomes. Teams one,two, three and five searched for and located appropriate equations establishing the association between the height of human beings andlength of femur bone. These equations were verified for their validity and then applied for computing values of the unknown variable ofvictim's height. Since the length of the femur of the victim had been given in the problem description as 55 cm, the final acceptable answerwas computed to be about 1.84 m. Team 4 decided not to use the equations found in Internet resource websites and conducted its ownexperiments by collecting data on the femur lengths and height measures of their classmates to establish the equational relationship

K. Laxman / Computers & Education 55 (2010) 513e526524

between these two constructs. Students from this team obtained the linear equation of H ¼ 2.0F þ 70. Applying the known value of thelength of femur into this equation, team four found the correct answer of 1.80 m.

Whenasked tocomment in their reflection journalsontheefficacyof theskills taughtduringbothphasesof the intervention trainingprograminaccomplishing the solvingof thisproblem, students generallyascribed their strongproblemsolvingperformance to theoverall effectivenessofthe instructional coaching imparted during the intervention training. Save for one student N, the rest of the students either categorically wrotethat theydidnotapply the2ndsetof skills involving framingofessential/foundationquestionsanddevelopingkeywordconceptmapsor failed toexplicitly make any reference to and comment on the utilization of these skills. Students reasoned that they found the 1st set of informationsearching skills to be more relevant and applicable in organizing their information searching in solving this well-structured problem scenario.Students explained that the 1st set of skills on learning about Internet search tools and information searching strategies were better alignedtowards the cognitive problem solving demands ofwell-structured problems. Task decomposition forwell-structured problem solving involvesan information search and analysis orientation that is typically limited, bounded and domain or disciplinary specific in scope. Hence, studentsfound that the elementary tier of knowledge on information search skills acquired in the first phase of the training interventionwas adequatelyrigorousenough insolving thegivenproblem. Leveraginguponthe2ndsetof advanced informationsearchskillswouldunnecessarilycomplicatethe solving of well-defined problems. Hence, there is a strong correlation between the 1st set of fundamental skills and successfully resolvingwell-structuredproblemsandaweakornon-existent impactof the2ndclusterof information search skillson theproductivityofwell-structuredproblem solving processes. In this regards, a sample of a student's reflections is as follows:

“I applied the first set of skills. If I did not apply the skills taught, I would have had difficulties in searching for the information I want andmay end up with a lot of unwanted information. This might slow down the process.” (student O)

4.7. Phase 6 problem (ill-structured)

All five teams performed credibly well in this ill-structured problem solving task and were able to successfully solve the problem. Thesefive teams had also extensively applied the principles of effective information searching skills taught during both phases of the interventiontraining in their information research pursuits in seeking solutions to the given problem. This was expressly evident from the grouppresentations and the analysis of artifacts submitted by these students e all five teams had developed a protocol of appropriate guidingessential and foundation questions and structured their information seeking activities by constructing keyword concept category maps. Thegood quality of students' products, particularly the structures of the concept maps and students' learning performances for the daymanifestly demonstrate the positive impact of the imparted training in improving students' information literacy and problem solvingmastery levels. All five teams listed the question of “What is a candle and what does it mean to different people?” or closely-related variants asthe key essential questions that framed their information research for this problem. The guiding foundation questions raised by studentscovered a number of thematic concepts such as materials, uses, history, colours and symbolizations.

In response to the reflection trigger on the problem solving utility of both sets of the taught Internet information searching skills, twenty-two students assessed both sets of skills to be relevant and beneficial in effectively the given problem. They averred that efficacious infor-mation searching skills are an important function of successful problem solving. The few students who did highlight difficulties encounteredin the application of these skills (especially the second set on developing essential and foundation questions and creating keyword categoriesmind-maps) noted that they arose primarily out of students' unfamiliarity and lack of exposure in exercising these skills to accomplishproblem solving. These studentswere of the opinion that givenmore opportunities for varied, distributed practice in honing these skills, theywould becomemore competent in the utilization of the skills. Some sample favourable comments drawn from the data corpus demonstratingthe positive impact of the instructed skills in enhancing ill-structured problem solving performance are attached herein for reference:

“I can easily say that these skills are essential to problem solving. I believe the skills helped us to easily complete the task and improvemycompetency in searching for information in the Internet. Coming up with themindmap enabled me to brainstorm for ideas before doingany research.” (student F)“All the skills I have learnt have been very useful. I will be using these skills in my daily information searching activities. This is animportant knowledge I have gained.” (student H).

All students in the five groups indicated in their reflection journal entries that the use of the 1st set of basic information searching skillssuch as applying Boolean operators and quotationmarks positively facilitated the solving of the ill-structured problem. In addition, they hadalso widely leveraged upon the techniques of essential and foundation questions crafting and keyword concept mapping e competenciesthey had learnt during the 2nd phase of the intervention training. They found these skills to be particularly valuable and effectual inaddressing the requirements of the given ill-structured problem task. These strategies were assessed to be powerful cognitive aids inscaffolding their problem solving activities. Student S succinctly summarized thus the learning gains from applying these questioning andconcept mapping strategies to solving ill-structured problems:

“Today before doing Internet information searching, we did the concept mind map. The mind map was organized according to theessential, foundation and sub-foundation questions we had asked ourselves. We derived the foundation questions from essentialquestions and the sub questions from the foundation questions. After that we chose the key words from the questions and created themind map. Then we proceeded to Internet searching.”

4.8. Impact of information searching skills on problem solving

In summary, a close analysis of students' solution presentations, artifacts and reflection journal responses for the six problem-based learning lessons (three well and three ill-structured problems) reveal the following nature of the relationships betweendifferent Internet information searching skills/strategies and the structuredness of problem tasks as outlined in the following sche-matic framework (Fig. 4).

Well-structuredProblems

Learning about Internet

Search Tools

Using Information Search

Strategies and

Techniques

Ill-structuredProblems

Learning about Internet

Search Tools

Using Information Search

Strategies and

Techniques

Framing Essential and

Foundation Research

Questions

Developing and

Organizing Search

Keywords

Fig. 4. Schematic representation of impact of information search skills on problem solving.

K. Laxman / Computers & Education 55 (2010) 513e526 525

5. Conclusions

This study has established that information searching skills do have an influence on problem solving performance. To take full advantageof problem-based learning, students have to use skills requiring them to become proficient users and organizers of information resourcesfound in many different locations and formats.

This study has affirmed that students need to examine the nature of a given problem and determine its attributes before definingproblem solving strategies and generating possible solutions. It was ascertained that the information needs of well-structured problemstend to be constrained, discipline-specific with related information searching to be done in seeking answers easily manageable. Beingconversant with fundamental information searching skills involving awareness of various Internet search tools as well as basic informationsearching techniques is sufficient for solving well-structured problems. On the other hand, the information demands of ill-structuredproblems are both multi-disciplinary and extensive. To successfully solve ill-structured problems, students need to apply more advancedinformation searching skills involving framing of essential and foundation research questions as well as developing and organizing searchkeywords.

This study examined the treatment effects of a set of information searching skills and strategies upon students' problem solvingperformance and outcomes. In this study the problem themes that were used were broadly classified as either well or ill-structuredproblems. Further research needs to be conducted to investigate the influencing impact of these information searching skills on each of thedifferent problems typeswithin thewell and ill-structured problem classification typology delineated by Jonassen.Well-structured problemtypes include logical, story and rule-using problems whereas ill-structured problem types include troubleshooting, diagnosis-solution andsituated case problems. The mediating influence of information searching skills in engaging in solving each of these specific problem typeswarrants further research. Associated problem solving strategies that students could apply to successfully deconstruct and solve each ofthese problem types also requires further investigation.

K. Laxman / Computers & Education 55 (2010) 513e526526

Appendix A: Pre-intervention survey

1) How did you learn to use the Internet?2) Do you plan your search before actually carrying it out in the Internet? If so, how do you do it?3) Which search tools do you use to find information in the Internet? Give reasons in explaining your choices.4) When using search engines, which search engines do you normally prefer to use? Why so?5) Do you use multiple search engines in doing your information searching? Why so?6) Do you think there are any similarities/differences in the ways different search engines work? What would they be?

References

Anderson, J. R. (1980). Cognitive psychology and its implications. New York: Freeman.Bates, M. J. (1979). Information search tactics. Journal of the American Society for Information Science, 30, 205e214.Burchinall, L. G. (1976). The communications revolution: America's third century challenge'. Texas.Eaglet on, M., Guinea, K., & Langlauf, K. (2003). Teaching Internet literacy strategies: the hero inquiry project. Voices From the Middle, 10(3), 28e36.Jakes, D., Pennington, M., & Knodle, H. (2002). Using the Internet to promote inquiry-based learning. Retrieved 21.04.05, from. http://www.biopoint.com/inquiry/ibr.html.Jonassen, D. H. (1997). Instructional design models for well-structured and ill-structured problem-solving learning outcomes. Educational Technology, Research and Devel-

opment, 45(1), 65e94.Jonassen, D. H. (2000). Toward a design theory of problem solving. Educational Technology, Research & Development, 48(4), 63e85.Jones, D. (1998). Exploring the Internet using critical thinking skills: A self-paced workbook for learning to effectively use the Internet and evaluate online information. New York:

Neal-Schuman.Landow, G. (1992). Hypertext: The convergence of contemporary critical theory and technology. Baltimore: Johns Hopkins University Press.Mayer, R. E., & Wittrock, M. C. (1996). Problem-solving transfer. In D. C. Berlinert, & R. C. Calfee (Eds.), Handbook of educational psychology (pp. 47e62). New York: Macmillan.McKenzie, J. (1996). Information fad bandwagon Internet world wide web. From now on Retrieved 22.04.05, from. http://fromnowon.org/sept96/questions.html.Nachiamas, R., & Gilad, A. (2002). Needle in a hyperstack: searching on the world wide web. Journal of Research on Technology in Education, 34(4), 475.Savery, J. (2006). Overview of problem-based learning: definitions and distinctions. The Interdisciplinary Journal of Problem-based Learning, 1(1), 9e20.Schacter, J., Chung, G. K. W. K., & Dorr, A. (1998). Children's Internet searching on complex problems: performance and process analyses. Journal of the American Society for

Information Science, 49, 840e850.Teixeira-Dias, J., de Jesus, P., de Souza, N., & Watts, M. (2005). Teaching for quality learning in chemistry. International Journal of Science Education, 27(9), 1123e1138.Vansickle, S. L. (2000). Tenth graders' search knowledge and use of the world wide web. Georgia State University.Wood, P. K. (1983). Inquiring systems and problem structures: implications for cognitive development. Human Development, 26, 259e265.Zurkowski, P. (1974). The information service environment: Relationships and priorities (No. ED 100391). Washington DC: National Commission on Libraries and Information

Science.