Paper ID #8856

African American High School Students’ Human-Centered Approach to De-sign

Mrs. Stacie LeSure Gregory, Utah State University

Stacie is a PhD candidate in Engineering Education at Utah State University. Stacie has a BS in Physicsfrom Spelman College and a MS in Materials Science and Engineering from Georgia Institute of Tech-nology. Stacie’s current research interests include the integration of Human-Centered Design and ServiceLearning opportunities to recruit and retain underrepresented students in engineering. She is also inter-ested in developing intervention strategies to reduce the negative consequences of Stereotype Threat whichmay be contributing to the lack of persistence of female and minority students in engineering education.

Prof. Nathan Mentzer, Purdue University, West Lafayette

Nathan Mentzer is an assistant professor in the College of Technology with a joint appointment in theCollege of Education at Purdue University. He prepares Engineering/Technology candidates for teacherlicensure. Dr. Mentzer’s educational efforts in pedagogical content knowledge are guided by a researchtheme centered in student learning of engineering design thinking on the secondary level. Nathan wasa former middle and high school technology educator in Montana prior to pursuing a doctoral degree.He was a National Center for Engineering and Technology Education (NCETE) Fellow at Utah StateUniversity while pursuing a Ph.D. in Curriculum and Instruction. After graduation he completed a oneyear appointment with the Center as a postdoctoral researcher.

Prof. Kurt Henry Becker, Utah State University - Engineering Education

Kurt Becker is a Professor in the Department of Engineering Education at Utah State University and thecurrent director for the Center for Engineering Education Research (CEER) which examines innovativeand effective engineering education practices as well as classroom technologies that advance learningand teaching in engineering. He is also working on a National Science Foundation (NSF) funded projectexploring engineering design knowing and thinking as an innovation in STEM learning. His areas of re-search include engineering design thinking, adult learning cognition, engineering education professionaldevelopment and technical training. He has extensive international experience working on technical train-ing and engineering projects funded by the Asian Development Bank, World Bank, and U.S. Departmentof Labor, USAID. Countries where he has worked include Armenia, Bangladesh, Bulgaria, China, Indone-sia, Macedonia, Poland, Romania, and Thailand. He is currently a consultant on a USAID-funded projectthat involves workforce development and enterprise competitiveness. He received his PhD from TexasA&M University in 1988 and teaches undergraduate and graduate courses in the engineering educationdepartment.

c©American Society for Engineering Education, 2014

Page 24.146.1

1

African American High School Students’ Human-Centered Approach in

Design

“While most people spend 95% of their time interacting with the technologies of the human-

made world, few know these products are made through engineering” 1, or that engineering

design is “the missing link that connects science and math with innovation” 1. Design is

recognized as the critical element of engineering thinking which differentiates engineering from

other problem solving approaches 2.

Engineering design has the potential to integrate science, technology and mathematics concepts

for students and is essential for developing technological literacy 3. For over a decade, experts

have been calling for a push to increase technological literacy of our Nation’s K-12 students 4-8

.

“The key to educating students to thrive in a competitive global economy is introducing them

early to the engineering design skills and concepts that engage them in applying their math and

science knowledge to solve real problems” 1.

While a demand for technological literacy is loud and clear, many young people are unprepared

to make informed decisions regarding the development of new technologies and their

applications. “ The relationship between understanding engineering and technological literacy is

of special urgency during the high school years, since “technologically literate people should

also know something about the engineering design process” 8. “Technology is the outcome of

engineering; it is rare that science translates directly into technology, just as it is not true that

engineering is just applied science” 9.

Today, Science, Technology, Engineering, and Mathematics (STEM) education continues to be a

national concern in the United States. Technology and engineering education (the ‘T’ and ‘E’ of

STEM) have seen increased attention in recent years. The National Academy of Engineering

commissioned a study titled “Engineering in K12 Education” which included a review of U.S.

curriculum materials related to the T and E of STEM education as well as the relationship

between Science, Technology, Engineering and Mathematics education. The National

Academy’s work emphasized the role of engineering in improving STEM education as it may be

a “catalyst” serving to draw connections between mathematics, science and technology education 10

and creative problem-solving.

Design Thinking

Design thinking is a creative way of problem-solving 11

. It promotes developments of diverse

ideas, which are essential for innovation 12

. Studies show that teaching design thinking helps

students in learning core subjects as well as fostering social skills 13-14

. In addition, it also

encourages students’ metacognition 15

.

In the 1990’s, design thinking gained popularity as a way to foster and sustain innovation by

having work environments that focused on the customer while simultaneously supporting

employee’s experimentation 16

. Design thinking is used in all industries from mechanical

Page 24.146.2

2

engineering, business, and healthcare to education, art, and design

(http://www.byui.edu/clusters/design-thinking).

Specifically, design thinking is defined as a human-centered design process to solving ill-

structured problems using an organized method of defining the problem by observing and

empathizing with the people who are impacted by it, generating multiple solutions, prototyping

one of the solutions, and then testing it 17

.

The CEO of IDEO, a company recognized as the leader in design thinking, has said that design

thinking might just be the quintessential career and college skill set for this new era, central to

success in every career 18

. In 2005, IDEO introduced design thinking to engineering students at

Stanford University with the confidence that engineers and scientists can be trained to become

innovators 19

.

Since design is such a predominant and characteristic activity of the engineering profession,

ABET has included it as an important criteria to evaluate engineering degree programs for

accreditation 20

. With design skills being so significant to future engineers, it is important that

graduates are prepared to conquer the design challenges they will encounter. Engineers must be

equipped with the tools and knowledge to participate in a globally competitive technological

society. In order to compete, engineers need to have skills that are current and relevant in this

age of innovation and rapid advances in technology.

According Krippendorff 21

, a paradigm shift is occurring in design from “technology-centered

design” to “human-centered design”. Therefore current and future engineering students must

acquire an understanding of how to design products, systems and services that meet or exceed

the needs, expectations, and requirements of the user. Tim Brown, the CEO and president of

IDEO, has encouraged engineering programs to develop the “design thinking” of their graduates

to ensure their readiness to compete and make an impact globally. In an article published in the

Harvard Business Review, Brown defined “design thinking” as:

a methodology that imbues the full spectrum of innovation activities with a human

centered design ethos. By this I mean that innovation is powered by a thorough

understanding, through direct observation, of what people want and need in their lives

and what they like or dislike about the way particular products are made, packaged

marketed, sold, and supported 22

.

In that same article, Brown stated

I believe that design thinking has much to offer a business world in which most

management ideas and best practices are freely available to be copied and exploited.

Leaders now look to innovation as a principal source of differentiation and competitive

advantage; they would do well to incorporate design thinking into all phases of the

process22

.

Like Brown, others concur that design thinking is an important element to promote innovation.

Vand deems design thinking as a creative way of problem-solving 23

. Likewise, Staw regards it

as a means to promote the development of diverse ideas, which are essential for innovation 24

.

Despite its importance, teaching design in a way to promote and enhance design thinking of

students poses many challenges to educators. According to Evans et al.:

Page 24.146.3

3

The subject [of design] seems to occupy the top drawer of a Pandora’s box of

controversial curriculum matters, a box often opened only as accreditation time

approaches. Even ‘design’ faculty—those often segregated from ‘analysis’ faculty by the

courses they teach—have trouble articulating this elusive creature called design 25

.

Human-Centered Design (HCD)

Human-centered design (HCD) also known as participatory design, reflective design, and

cooperative design, is a design approach which aims to actively involve the end users in the

design process. The goal of HCD is to ensure that products are not only usable, but also designed

to meet the needs of the users. According to IDEO, Human-centered design is:

a process and a set of techniques used to create new solutions for the world. Solutions

include products, services, environments, organizations, and modes of interaction. The

reason this process is called “human-centered” is because it starts with the people we are

designing for. The HCD process begins by examining the needs, dreams, and behaviors

of the people we want to affect with our solutions. We seek to listen to and understand

what they want. We call this the Desirability lens. We view the world through this lens

throughout the design process. Once we have identified a range of what is Desirable, we

begin to view our solutions through the lenses of Feasibility and Viability. We carefully

bring in these lenses during the later phases of the process26

.

Zhang and Dong 27

summarized the following characteristics of human-centered design:

1. “The central place of human beings”

2. “Understanding people holistically”

3. “Multi-disciplinary collaboration”

4. “Involving users throughout the design process”

5. “Making products or services useful, usable, and desirable”

Human-centered design directly contrasts technology-centered design. As stated by Krippendorf

technology-centered design

improves the world in the designers’ or their clients’ terms. Making a machine cheaper to

produce, more energy efficient, or more usable by more people may well be intended to

and actually does benefit a community of users, but the measures of these benefits are the

designers’ choice…imposed from above, by experts onto lay people 21

.

Human-centered design considers all aspects of the technical, organizational and physical

environments while focusing on the physical abilities and physical needs of the user. HCD is the

practice of designing and developing products and services, buildings, and entire communities

while taking into consideration information about the user. HCD relies on research and data

regarding the cognitive and physical capacities, limitations, social needs, and task requirements

that empowers all users to function at their highest capacity, regardless of age or ability

(http://www.aging.ny.gov/LivableNY/ResourceManual/Index.cfm).

Like other organizational practices, HCD must maintain standards in order to be effectively

implemented. International Organization for Standardization (ISO) created a HCD standard

(http://www.iso.org/iso/home/about.htm. These standards detail an iterative development cycle

where product specifications take into account the requirements of the user and organization, as

Page 24.146.4

4

well as, specify the context in which a product is to be used. The standards include Four

Principles of Human-Centered Design:

1. active involvement of users

2. appropriate allocation of function to system and to user

3. iteration of design solutions

4. multidisciplinary design

and Four Human-Centered Design Activities:

1. understand and specify the context of use

2. specify user and organizational requirements

3. produce more than one candidate design solution

4. evaluate designs against requirements

Brown contends that the human-centered approach to design is a recognized contributor to

innovations in engineering design 22

. HCD also assists students in enhancing skills such

creativity, practical ingenuity, and communication all which are essential for the Engineer of

2020 (National Academy of Engineering 27

. In addition to equipping engineering graduates with

required skills to compete globally, HCD addresses the Grand Challenges identified by the

National Academy of Engineering 28-29

.

“While many institutions are pursing the principles of HCD through service-learning and

sustainability engineering design challenges, there are many open questions concerning human-

centered design learning ” 30

. Effectively teaching human-centered design poses many

challenges as it requires access to users and stakeholders. Nonetheless, service-learning is a

viable context for teaching students the principles of human-centered design. According to

Zoltawski,“Service-learning, a growing pedagogy within engineering, offers many synergistic

opportunities to create a human-centered design experience” 31

.

Service Learning

Teaching human-centered design within a service-learning context provides several benefits.

Students are equipped with the essential tools to enhance their innovation and design thinking

skills. Additionally, communities receive benefits by having pertinent needs met at little or no

cost to them. Service-learning may also prove to be invaluable in the recruitment and retention

of under-represented students in engineering.

Service learning is “a type of experiential education in which students participate in service in

the community and reflect on their involvement in such a way as to gain further understanding of

course content and of the discipline and its relationship to social needs and an enhanced sense of

civic responsibility” 32

.

As stated by Hatcher and Bringle,

Service learning extends the classroom into the community, and students frequently

encounter unfamiliar situations that challenge and contradict their perspectives. Real

world issues (e.g., crime, homelessness, illiteracy and poverty) provide rich opportunities

for students to reconsider their values in light of their own and other students’ service Page 24.146.5

5

experiences. Values are presumed to guide decisions. As values are explored, clarified,

and altered, it would be expected that a student’s behavior would be modified 33

.

Its commitment to and its potential to clarify values related to social responsibility and civic

literacy are characteristics which distinguish service learning from other types of experiential

education 34

.

According to Astin et al.,

Service learning represents a potentially powerful form of pedagogy because it provides a

means of linking the academic with the practical. The more abstract and theoretical

material of the traditional classroom takes on new meaning as the student “tries it out,”

so to speak, in the “real” world. At the same time, the student benefits from the

opportunity to connect the service experience to the intellectual content of the classroom 35

.

Results from a study conducted by Astin et al. 35

showed that service participation yielded

significant positive results in the following eleven areas: (1) GPA, (2) writing skills, (3) critical

thinking skills, (4) commitment to activism, (5) commitment to promoting racial understanding,

(6) self-efficacy (7) leadership activities (8) self-rated leadership ability, (9) interpersonal skills,

(10) choice of a service career, and (11) plans to participate in service after college.

A joint investigation at the University of Massachusetts-Lowell and the Massachusetts

Institute of Technology disclosed that students who participated in service-learning had a better

understanding of the connection between engineering and community needs 36.

According to

Tsang et al., “Service learning is an effective strategy to enable engineering schools to attain the

objectives outlined in recent reports on reforming the undergraduate engineering curriculum for

the 21st Century” 37

.

While engineering has been more resistant than other disciplines to adopt service-learning, there

is evidence of increased interest in this pedagogical approach within engineering. Both curricular

and extra-curricular models of service-learning have been implemented. The EPICS Program,

originally launched at Purdue University has been adopted at several universities. Engineers

without Borders, Engineers for a Sustainable World and Engineers for World Health are notable

examples of extra-curricular models of service learning.

Human-Centered Design and Context-of-Use

Every product, service or system developed will be used within a certain context and by users

with particular characteristics. They will have specific goals and distinctive tasks to perform. The

product, service or system will also be used within a distinct range of technical, physical, social

or organizational conditions that may affect its use. The quality of use, including usability, as

well as, user health and safety, depends on having a sound grasp of the context of use 38

.

Page 24.146.6

6

As depicted in Figure 1, the first step of HCD is to specify and understand a product (or service)

context of use. Context of use includes 38

:

the goals of the user community

the goals of the main user

task characteristics

environmental characteristics

Context-of-use analysis developed by Allison et al. is a structured method for eliciting detailed

information about the context of use for stability activities, particularly user requirements,

specification and evaluation 39

. Context-of-use analysis is an outcome of the ESPRIT HUFIT

project (Human Factors in Information Technology). It was developed for identifying user

types, their needs and characteristics, and translating this information into user requirements 43

.

Knowledge of the context-of-use improves the overall design of a product. This information

guides the assessment of usability by providing a structured analysis of the relevant

characteristics of the intended users, tasks and environments for the product is being

developed 38

.

Method: Data Collection

This research investigates how African-American high school students apply HCD thinking to

open-ended design. Data used for this study were gathered as part of a larger NSF funded

DRK12 study titled, Exploring Engineering Design Knowing and Thinking as an Innovation in

STEM Learning (DRL-0918621). In this work, Becker, Mentzer and Parks 40

identified four high

schools which offer a series of courses on engineering design. The schools, located throughout

the United States, are representative of both rural and urban regions. Exemplary students who

specify and understand the context of use

specify user and organizational requirements

produce more than one candidate design solution

evaluate designs against requirements

Figure 1: Human-Centered Design

Page 24.146.7

7

were finishing the sequence of engineering design courses were recruited to participate in this

study. Each student completed a design problem consistent with previous literature 41-43

.

Students worked individually for up to three hours, using a ‘think aloud’ protocol to develop a

solution to the design problem. Verbal protocol is a tool used by researchers 44-49

in a variety of

fields including engineering and technology to document student design processes. It is a

method which can provide an “in-depth understanding of the processes students use to solve

engineering design problems” 50

. Ericsson and Simon suggested a three step approach to

conducting verbal protocol: recording, transcription/segmenting, and coding into categories.

“This is a research method in which subjects think aloud as they solve problems or perform a

task. The subjects’ thought processes are captured on audio and/or videotape ” 51

. According to

Ericsson and Simon, “The concurrent [verbal] report reveals the sequence of information that is

heeded by the subject without altering the cognitive processes, while other kinds of verbal

reports may change these processes” 51

. Consistent with previous literature, sessions for this

research was video and audio recorded and paper based artifacts were gathered.

The design task presented to students was similar to previous work 41-43

and included these

instructions:

You live in a mid-size city. A local resident has recently donated a corner lot for a

playground. Since you are an engineer who lives in the neighborhood, you have been

asked by the city to design a playground.

Any equipment you design must

• be safe for the children

• remain outside all year long

• not cost too much

• comply with the Americans with Disabilities Act

The neighborhood does not have the time or money to buy ready-made pieces of

equipment. Your design should use materials that are available at any hardware or lumber

store. The playground must be ready for use in 2 months.

At the conclusion of the problem statement, students were prompted that additional information

is available about the problem including a lot diagram and they could ask for it now (see figure

2). This practice request was unique in that students were not prompted again to request

information, but positioned at the start of the problem to demonstrate the process of asking for

information and that information was, in fact, available as stated. It was assumed that once

students understood they could ask for information, they would feel comfortable asking for what

they thought they needed while they worked. When students asked the administrator for

information, they were either provided the information requested, told the administrator did not

have it or asked to be more specific. The administrator’s response to an information request was

first to acknowledge that they understood and then look in the packet of information. They would

look through the information available even if they knew that the requested information was not

available so that students would not feel they were off target. Administrators were friendly and

welcoming of students to be more specific prior to providing information. Students would

sometimes ask for very general information such as “what information do you have about wood

chips”. The administrator would respond, please be more specific. The student would often

Page 24.146.8

8

respond that they wanted the cost of wood chips or they wanted to know how deep wood chips

needed to be to be safe or how far around a slide wood chips should be placed to protect children

or the longevity of using wood chips as compared to rubber mulch. Refer to figure 1 for a

sample.

The administrator would continue the problem until the participant indicated they were finished

(or the three hour session had expired). Once the participant felt that they had completed a design

that satisfied the problem, the administrator would thank them for participating in the study and

remind them that there will be a follow up interview in a few weeks. Follow up interviews were

usually conducted 2-4 weeks after the initial design task was completed. These served as a way

for the research team to gain more information about what student were doing while developing

their solution. Common questions asked of participants were, how did you define the problem,

how did you compare ideas, why and how did you choose your final idea or plan, along with

questions directly related to the students work.

Figure 2: Design Problem Lot Diagram

Method: Data Analysis

This research analyzed the following artifacts:

Video/audio recordings

1. time to complete design challenge

2. paper-based artifacts, including student sketches/student work

3. follow-up interviews

Video/audio recordings, as well as, follow-up phone interviews were transcribed, by hand. The

students’ design work captured on paper was also reviewed. The transcribed data and student

sketches were then coded. Four different coding categories were used: (1) information requests;

(2) constraints; (3) expert design criteria and (4) context-of-use.

Information Requests

Page 24.146.9

9

The data were coded for “gathering information” as presented by Mosborg et al.52

.

The gathering information element of the design process was one of nine elements considered in

previous work (foundation work for the DRK12) and included students looking for information

to help them solve the problem. Coding included what information was requested by the

participant and at what point in time. Also consistent with prior research, the following

categories of information were available for participant request 52

: budget, information about the

area, material costs, neighborhood opinions, utilities, neighborhood demographics, safety,

maintenance concerns, labor availability and costs, legal liability, material specification,

supervision concerns, availability of materials, body dimensions, disabled accessibility, technical

references, and other information. Adopted from previous literature 52

, information requests

were coded into the categories listed in table 1, with two exceptions, one added by this research

team- “climate” and one added during the DRK12 study-“clarity”, from which this data was

taken.

Table 1: Information requests and statement

Request

Category

Statements Pertaining to:

activities at least 3 activities constraint

age 10 years of age constraint

body dimensions human body size(s)

budget amount of money available for the project

clarity making instructions or diagrams for the people building the playground; explain your

solution as clearly and completely as possible

climate weather/climate

demographics composition of the neighborhood population

dimensions specific measurements (typical, ballpark, or actual) of playground equipment, layout, or the lot

facilities playground facilities such as bathrooms, night lighting, or water fountains

handicapped safety or accessibility for persons with disabilities

labor workers for the project

legal liability for potential injuries or accidents

maintenance property or equipment maintenance for the playground’s operation

material cost of specific materials.

material cost and

budget

cost of specific materials with respect to budget or affordability.

material

specifications

technical material requirements

material type the general type of material needed (e.g., wood, 2x4’s, steel, screws, nails, paint)

neighborhood area the location of objects in the area surrounding the lot

neighborhood

conditions

other conditions of the area

occupancy “12 children kept busy” constraint

opinions stakeholders’ reactions to the proposed playground, or their preferences for equipment or

activities.

park area inside the

lot

lot’s characteristics or layout

safety “safe for children” constraint

schedule “ready in 2 months” constraint for constructing the playground equipment

supervision looking after children during playground hours

Page 24.146.10

10

supplier “use material available at any hardware or lumber store” constraint

technical reference technical construction requirements

utilities gas, water, or power lines.

Constraints

In previous studies 42

, there were seven constraints provided in the design problem. For this

study, two additional constraints were added. The nine constraints are listed in table 2.

Table 2: Description of Constraints

# Description of constraint

1 The design allows at least 12 children to be kept busy

2 There are at least 3 activities provided

3 Equipment can be used outdoor all year

4 Materials are available at any hardware or lumber store

5 Playground can be completed in two months

6 The cost of the playground does not exceed budget

7 An effort has been made to allow handicapped children to be able to use playground

8 Design is explained as clearly as possible. Someone can build it without any questions

9 The playground is safe

Expert Design Criteria

Moore, Goltsman and Iacofano 53

published documentation used to assess the safety of

playground designs including thirty-three criteria appropriate for all playgrounds. These criteria

(table 3) were included as part of a coding scheme to assess students’ human centered design

thinking.

Table 3: Expert Design Criteria

# Expert Design Criteria 1 Location of play areas allows minimum contact between children and traffic.

2 Entrances to the park are clearly identified. (i.e. sign)

3 Entrances to the park are visible from nearby housing.

4 Entrances to the park direct young pedestrians along safe routes through the park.

5 Parking areas are separated from play areas by barriers.

6 Parking area perimeters are open and unobstructed to view.

7 Play areas are accessible from main park pathways.

8 Play areas are accessible to one another.

9 Main pathways are connected with entrances and play areas.

10 Play areas are defined using fences, berms, or plants.

11 Access ways to play areas are at least 10 ft. wide and capable of supporting service vehicles.

12 Playground must be supervised at all hours it is open or have a sign posted in the lot that frees the city of all

responsibility if some accident were to occur.

13 Play area provides challenges to stimulate upper body strength, i.e. rings, turning bars, horizontal bars,

climbing trees, swinging ropes, jungle gym, and things to lift.

14 These challenges are designed and positioned to promote mixed use by children with and without

disabilities.

15 These challenges are designed to reduce the possibility of injury, especially protecting children from falling

and collision.

Page 24.146.11

11

16 Balance settings which stimulate the inner ear are provided, i.e. tire swings, climbing surfaces, bridges,

narrow rails, seesaw, or walls.

17 Coordination and judgment settings are provided, i.e. horizontal ladders, stepping logs, climbers, tunnels,

and banister slides.

18 Design provides at least three levels of difficulty for each activity (e.g. monkey bars with increasing width

between bars at different levels.)

19 Design provides at least three levels of accomplishment for each activity (e.g. using slides that are straight

and curved.)

20 Children may enter and exit a setting at intermediate levels.

21 Next challenge is apparent from previous challenge.

22 Challenges are NOT related to hazards and danger but are related to more difficult mastery of the body.

23 Play area contains physical elements that can be changed and moved around.

24 Fasteners used on play equipment are vandal-resistant and protrusions meet CPSC guidelines (e.g. recessed,

fitted with tamper-proof locks, and the holes

25 Play equipment over 24 inches high has an unobstructed fall zone with shock-absorbing surfacing (for

school age children, 20 in. for preschoolers).

26 The fall zone extends a min of 6 feet in all directions from the perimeter of the equipment.

27 Fall zones of adjacent pieces of equipment do not overlap.

28 Surfacing material attenuates the impact of a head first fall from the highest point of the equipment

29 There are no sharp points, corners, or edges that might puncture children’s skin.

30 Protrusions or projections must not be capable of entangling children’s clothing.

31 There are no accessible pinch, crush, or shearing points or exposed moving parts on playground equipment

that could injure children or catch their clothing

32 Components or groups of components do not have or form openings that could trap a child’s head (3.5 in <

opening < 9.0 in).

33 All devices that anchor playground equipment to the ground are installed below the playing surface.

Context-of-Use

As presented by Maguire, the main elements of a context analysis are described below 38

.

user goals and characteristics (UG):

The predominant part of the context-of-use analysis focuses on the user. If the user population

consists of more than one type of user, an analysis is completed for each user type.

tasks (TA):

The tasks are the various activities completed to achieve a goal. A description of the activities

and procedures involved in performing a task should be related to the desired goals to be

achieved by the user(s).

technical environment (TE):

Technical environment includes the software, hardware, and other required equipment used in

conjunction with the product. Characteristics of the technical environment may affect the

usability of the product.

physical environment (PE):

The physical environment can profoundly impact the usability of a product. Location of the

product can affect usability.

social or organizational environment (OE):

The way people work, the availability of assistance and the frequency of interruptions can affect

usability of a product.

Page 24.146.12

12

Sample

The sample population studied in this research is a subset of the population used in previous

work by Becker, Mentzer and Park 40

. Data was collected from four schools representing four

states encompassing both urban to rural regions (refer to table 4 and table 5 for school and

community demographic information). Criterion sampling strategy 54

was used and included:

The high schools had an established program of study which employed a focus on

engineering in a sequence of courses developed in association with an engineering

outreach effort as part of a university program.

In these courses, students participated in design activities which engaged their critical

thinking and problem solving skills within the framework of the engineering design

process.

Table 4: School Demographics

Table 5: Community Demographics by School

School Community

Population Median

Household

Income

African American

American Indian

Asian Caucasian Hispanic

1 91,000 $45,000 1.2% 0.5% 4.0% 88.3% 8.2%

2 78,000 $34,000 2.3% 1.2% 1.4% 79% 23.6%

3 61,000 $36,000 3.2% 0.4% 1.2% 88.9% 9.1%

4 >500,000 $59,000 54.0% 0.4% 3.2% 40.6% 8.8%

Source: http://quickfacts.census.gov/qfd/index.html

Three target student populations were used in this study. Data for thirty students including 10

African American males, 10 Females (various races) and 10 White males were analyzed. This

subset of the total population was chosen as a means to evaluate whether race and/or gender

impacted human-centered design thinking of high school students.

Results: Time to Complete the Design Challenge

The total time each group of students spent completing the design challenge is presented in table

6. The times listed represent the combined time for all 10 students in each group. African

American male students spent a total of nine hours, forty-five minutes and four second. The

female took a total of thirteen hours, three minutes and nine seconds to complete the challenge.

White males dedicated sixteen hours, fifty minutes and fifty seconds to the playground design

challenge. The White male students devoted the most time to the challenge. They spent seven

hours, six minutes and forty-six seconds longer to complete the design challenge than the

School Enrollment Female Male African

American

American

Indian

Asian Caucasian Hispanic

1 1136 45% 55% 2% 1% 3% 65% 30%

2 216 54% 46% 1% 1% 1% 76% 20%

3 1833 47% 53% 4% 1% 1% 86% 7%

4 874 55% 45% 96% 0% 1% 1% 2%

Source: http://nces.ed.gov/ccd/schoolsearch/index.asp

Page 24.146.13

13

African American Students. The time the White male students spent on the playground design

challenge exceeded the female students’ time by three hours, forty-seven minutes and eleven

seconds.

Table 6: Total time for students to complete playground design challenge

Results: Information Requests

All information requests made by the students was transcribed and coded into one of the

following information request categories.

Table 7: Information Requests

Information African Americans Females White Males Total

Activities 0 3 6 9

Age 0 0 0 0

Body Dimensions 0 1 4 5

Budget 2 4 1 7

Clarity 3 4 3 10

Climate 1 1 3 5

Demographics 8 2 0 10

Dimensions 0 0 2 2

Facilities 0 0 0 0

Handicapped

Accessibility

6 6 7 19

Labor 1 3 1 5

Legal 0 0 0 0

Maintenance 0 1 0 1

Material Cost 3 26 26 55

Material Cost and

Budge

1 2 8 11

Material

Specifications

1 1 3 5

Material Type 2 1 2 5

Neighborhood Area 2 1 1 4

Neighborhood

Condition

0 0 0 0

Occupancy 0 0 1 1

Opinions 0 1 0 1

Other 0 0 1 1

Park Area Inside the

Lot

7 5 5 17

Safety 0 1 1 2

Schedule 0 0 0 0

Supervision 0 0 0 0

Supplier 1 0 1 2

Technical Reference 0 4 9 13

African American Males Females White Males Total Time

9:44:04 13:03:39 16:50:50

39:38:33

Page 24.146.14

14

Utilities 0 0 0 0

Total 38 67 85 190

There are a total of twenty-nine different information request categories. Seven categories

received no requests from any of the thirty participants. These seven categories are: age,

facilities, legal, neighborhood condition, schedule, supervision and utilities. There were a total of

190 information requests from all 30 students. Material cost was the top information request

category when combining all 3 student groups. Twenty-nine percent of the total number of

information requests was in the material cost category. However, of the information requested

by African American males, only 8% was for material cost; whereas, 31% of the information

requested by White males was material costs. 80% of the demographic information requested by

all students came from African Americans males. Of all the information requested by African

Americans, 21% was demographics. White males requested no demographic information and of

the information requested by females, only 3% were in the demographic category. 69% of all the

technical reference information requested by all students came from White Males; African

Americans requested no technical reference information. Handicapped Accessibility was the

second most requested category for all students. 10% of all the information requested by all

students was in this category. 16% of the information requested by African Americans was in the handicapped accessibility category; 9% for females and 10% for White males.

Table 8: Top 3 categories requested by the students

Two of the top three categories requested by the African American male participants were

demographics and handicapped accessibility (see table 8). Two of the top three categories

requested by White males were materials costs and technical reference. Although handicapped

accessibility was also one of the top three information categories requested by White males, the

percentage of handicapped accessibility was only 10% of the total requests made; whereas, for

African American males, handicapped accessibility requests comprised 16% of the total requests

made by these students. Based on accepted definitions of human-centered and technology-

centered design processes, it seems appropriate to consider demographics and handicapped

accessibility as human-centered; whereas, technical reference and material cost are more

technology centered.

The constraints, expert design criteria and context-of-use statements made on the day of the

design challenge, as well as, any relevant comments made during the follow-up interview were

coded. Additionally, for constraints and expert design criteria, the actual design solution the

students sketched while completing the challenge were analyzed and pertinent information was

coded. For the context-of-use, only transcribed statements made on the day of the challenge and

All Students African Americans Female White Male

IR Category % IR Category % IR Category % IR Category %

material cost 29 demographics 21 material cost 39 material cost 31

handicapped

accessible

10 park area 18 handicapped

accessible

9 handicapped

accessible

10

park area inside

the lot

9 handicapped

accessible

16 park area inside

the lot

7 technical

reference

11

Page 24.146.15

15

during the follow-up interviews that were pertinent to one of the five categories (UG, TA, TE,

PE, OE) were coded. In some cases, statements that were coded as context-of-use may have also

been coded in one of the other three categories: information requests, constraints, and expert

design criteria. If a statement was pertinent to one of the five context-of-use categories, it was

coded accordingly. The context-of-use results are meant to provide meaningful information

regarding how students intuitively think about design from a human-centered approach.

As shown in table 9, the top three constraints considered most frequently by the students are: C8,

C2, and C6 which are: “Design is explained as clearly as possible. Someone can build it without

any questions.”; “There are at least 3 activities provided.”; and “The cost of the playground

does not exceed budget”, respectively. 19% of all transcribed statements were coded for C8,

which refers to how clearly the students described their design so that they could be built by

someone without questions. 17% of all the statements made by all 3 groups of students pertain

to the constraint of including at least 3 different activities. 16% of all transcribed statements

were coded as C6: The cost of the playground does not exceed budget.

Table 9: Results (Constraints)

# Constraints African

American

Males

Females White Males All

Students

C1 12 children 8 36 26 70

C2 3 activities 19 68 77 164

C3 Equipment used year-round 24 11 19 54

C4 Materials at hardware store 9 46 34 89

C5 Completed in 2 months 6 3 16 25

C6 Cost does not exceed

budget

20 89 47 156

C7 Allows for handicapped

children

38 16 36 90

C8 Design explained clearly 3 55 115 173

C9 Safe 57 35 42 134

Total 184 359 412 955

As shown in table 10, African American males, the top three constraints considered are C9, C7

and C3. 31% of the African Americans transcribed statements references the safety of the

playground. 21% of their statements took into the constraint that the playground should allow

handicapped children to play. 13% of the statements made by African American students were

coded for the equipment scan be used outdoor all year. 25% of the statements made by the

female students were code for C6: ‘The cost of the playground should not exceed the budget.

The top constraint considered by the White male students was C8:” Design is explained as

clearly as possible. Someone can build it without any questions.” In agreement with the

information request results, the constraints most accounted for by African Americans were more

“human-centered”: Safety and Handicapped Accessibility. Whereas with the White males, the

constraints most considered with C8, C2, C6 which refer to how well the design is explained, the

activities provided and the budget are more “technology-centered”.

Page 24.146.16

16

Table 10: Top 3 Constraints

All Students African American

Males

Females White Males

Constraint % Constraint % Constraint % Constraint %

C8 18 C9 31 C6 25 C8 28

C2 17 C7 21 C2 19 C2 19

C3

16 C3 13 C8 15 C6 11

All the expert design criteria are human-centered. As shown in table 11, there are some that are

more focused on the user group, including safety and handicap accessibility; while others are

more focused on the actual tasks kids can participate in on the playground. Some of the expert

design criteria are more central to the actual physical appearance of the playground.

Table 11: Expert Design Criteria

All Students African American

Males

Females White Males

edc # % edc # % edc # % edc # %

13 & 16 11

15 15 16 13 13 &16 15

8 & 10

10

8, 10, & 21 12

8 & 10

11

14 &15 9

20 9

12 10 12 & 21 9 8, 10 &

21

7

The top design criteria considered by all students are:

EDC 13 (11%): Play area provides challenges to stimulate upper body strength, i.e. rings,

turning bars, horizontal bars, climbing trees, swinging ropes, jungle gym, and things to

lift.

EDC 16 (11%): Balance settings which stimulate the inner ear are provided, i.e. tire

swings, climbing surfaces, bridges, narrow rails, seesaw, or walls.

EDC 8 (10%): Play areas are accessible to one another.

EDC 10 (10%): Play areas are defined using fences, berms, or plants.

EDC 20 (9%): Children may enter and exit a setting at intermediate levels.

The expert design criteria considered most by the African American students is EDC 15 which is

related to the safety of the children. EDC 15 states: “These challenges are designed to reduce

the possibility of injury, especially protecting children from falling and collision.” This is

clearly human-centered, with a strong emphasis on safety.

Page 24.146.17

17

The top expert design criterion the female students considered was EDC 16: “Balance settings

which stimulate the inner ear are provided, i.e. tire swings, climbing surfaces, bridges, narrow

rails, seesaw, or walls”.

White males considered EDC 13 and EDC 16 most frequently: “Play area provides challenges to

stimulate upper body strength, i.e. rings, turning bars, horizontal bars, climbing trees, swinging

ropes, jungle gym, and things to lift”; and “Balance settings which stimulate the inner ear are

provided, i.e. tire swings, climbing surfaces, bridges, narrow rails, seesaw, or walls.”

The expert design criteria considered by both the female and the White male students focus more

on the actual task (Tire swings, climbing surfaces). African American male students accounted

for expert design criteria that speaks directly to the safety of the user. African American males

consistently focus on the user group.

As shown in table 12, African American male students made 82% more statements than the

white male students that were coded for context-of-use, the first step in the human-centered

design process. Moreover, these same students made 60% more context-of-use statements than

the female participants.

In total, seven-hundred and thirteen statements were coded for context-of-use. 46% of these

statements were made by African American students; 29% came from females and White males

made 25% of these statements.

Table 12: context-of-use

Human Centered

Design

African American

Males Females White Males

All Students

Code # % # % # % # %

UG 132

40.2 74

36.1 69

38.3 275 38.6

TA 71 21.6 61 29.8 64 35.6 196 27.5

TE 6 1.8 28 13.7 11 6.1 45 6.3

PE 83 25.3 41 20.0 36 20.0 160 22.4

OE 36 11.0 1 0.5 0 0.0 37 5.2

Total 328 205 180 713

Although, White male students spent seven hours, six minutes and forty-six seconds longer to

complete the design challenge than the African American Students, they made fewer “statements

relating to human-centered design, specifically, context-of-use. White males made only 180

context-of-use statements, compared to 328 such statements a made by African American males.

User group (UG) was the largest context-of-use category coded. 40% of the 328 statements

made by African American males and coded as context-of use were for UG; whereas for females

and White males, UG accounted for 36% and 38%, respectively. The second largest category

Page 24.146.18

18

under context-of-use was task (TA). 27.5% of all the statements made were coded in this

category. 35% of all the statements made by white males and coded as context-of-use belonged

to this category. TA accounted for 21% of the statements made by African American males and

30% of those made by females.

The least coded categories in context-of-use were for technical environment (TE) and

organizational environment (OE). While 13.7% of the statements made by females were coded

under TE, the male students’ comments accounted for 6.1% (White) and 1.8% (African

American). OE, the least coated category, accounted for 11% of the statements made by African

American males and 0.5% of those made by females. There were not transcribed statements

made by white males that were coded under OE.

Discussion

An important facet of design thinking is understanding the people impacted by design. The

literature is replete with examples that confirm this lack of understanding of the user, as well as,

an inadequate comprehension of how the product will be used. This knowledge deficit has

contributed to design failures 55-57

. Even though ABET acknowledges the significance of

engineering students attaining design thinking skills, knowing how to teach these skills continues

to eludes educators. As design is shifting from “technology-centered” to “human-centered”,

educators are now faced with the additional challenge of developing curriculum strategies that

encompass this change. Knowing how students innately think about human-centered design can

guide researchers, educators, and curriculum developers as they create meaningful and effective

educational tools for current and future engineering students.

As the nation is facing a shortage of qualified engineers, and fewer Americans are entering and

completing engineering degree programs, cultivating K12 students’ inherent design thinking

skills may prove invaluable in engaging students in engineering.

The results presented in this paper demonstrate that although high school students were not

explicitly taught human-centered design processes, they considered the user in a design

challenge. The four coding categories analyzed to measure the extent to which the students

applied human-centered design processes are: information request, constraints, expert design

criteria and context-of-use. The data indicate that the focus on the user is most apparent for

African American students. The top three categories of information requested by all students

were: (1) material cost, (2) handicapped accessibility and (3) park area inside the lot. For African

American students, the most requested information was demographics. White students, on the

other hand, requested more information regarding material cost. Analysis of the second coding

category, constraints, also highlights African American students’ tendency to focus on the user.

African American students’ top three constraints considered were playground safety,

handicapped accessibility and equipment usage year-round. While white students focused more

on the constraints related to the budget, the number of playground activities available and how

well the design was explained. This focus on the user continues for African American students

when analyzing expert design criteria. The number one expert design criteria African American

students accounted for is related to safety: “These challenges are designed to reduce the

Page 24.146.19

19

possibility of injury, especially protecting children from falling and collision.” On the other

hand, White students accounted most often for the design criteria “Balance settings which

stimulate the inner ear are provided, i.e. tire swings, climbing surfaces, bridges, narrow rails,

seesaw, or walls.” The final coding category analyzed in this study was context-of-use. African

American male students made 82% more statements than the white male students that were

coded for context-of-use. They also made 60% more context-of-use statements than the female

participants. In total, seven-hundred and thirteen statements were coded for context-of-use. 46%

of these statements were made by African American students; 29% came from females and

White males made 25% of these statements.

The total time to complete the design challenge was also an integral part of the data analysis.

Although, White male students spent on average seven hours, six minutes and forty-six seconds

longer to complete the design challenge than the African American Students, they made fewer

“statements relating to human-centered design, specifically, context-of-use. White males made

only 180 context-of-use statements, compared to 328 such statements a made by African

American males.

If African American students naturally approach design from a human-centered approach

researchers, educators and policy makers can use this information to devise best practices to

attract this demographics into engineering profession. Real-world, service learning projects

which afford these students the opportunity to apply their innate ways of thinking about design

should be furthered investigated.

Implications and Recommendations for Further Research

In order to confirm the results highlighted in this research, a larger scale study to evaluate the

human-centered design thinking of African American students is recommended. The data in this

research points to a trend suggesting African Americans natural inclination to consider the user;

thereby approaching design from a human-centered perspective. A real-world, design challenge

which pulls at the heart strings of African American students may be an effective way to engage

them in engineering. Research has shown that African American students are more drawn to

careers in which they can positively contribute to society. Demonstrating to this population of

students how engineering is human-centered may be promising.

Secondary educators can capitalize from the encouraging evidence that service learning projects

approached from a human-centered design perspective captures the attention of all students, but

especially those who are traditionally underrepresented in STEM. Such knowledge can equip

high school teachers with the tools they need as they face the challenge of preparing students for

STEM careers. Providing students with real-world design challenges relieves educators of the

burden of trying to create innovative ways to authentically teach engineering design.

Page 24.146.20

20

References

1. National Center for Technological Literacy. Our Nation's Challenge. 2010 [cited 2010 January 12];

Available from: http://www.mos.org/nctl/our_nations_challenge.php.

2. Dym, C. L., Agogino, A. M., Eris, O., Frey, D. D. and Leifer, L. J. (2005) Engineering design thinking,

teaching, and learning, Journal of Engineering Education 94(1): 103–120.

3. Katehi, L., G. Pearson, and M. Feder, eds. Engineering in K-12 Education. 2009, The National Academies

Press: Washington, D.C.

4. Garmire, E. and G. Pearson, eds. Tech Tally: Approaches to Assessing Technological Literacy. 2006,

National Academies Press: Washington, D.C.

5. Gorham, D., Engineering and Standards for Technological Literacy. The Technology Teacher, 2002. 61(7): p. 29-34.

6. International Technology Education Association, Technology for All Americans: A Rationale and Structure

for the Study of Technology. 1996, Reston, VA: Author.

7. International Technology Education Association, Standards for Technological Literacy: Content for the

Study of Technology. 2000, Reston, VA: Author.

8. Pearson, G. and A.T. Young, eds. Technically Speaking: Why All Americans Need to Know More About

Technology. 2002, National Academy of Engineering.

9. National Academy of Engineering, The Engineer of 2020. 2004, Washington, DC: The National Academies

Press.

10. National Center for Technological Literacy. Our Nation's Challenge. 2010 [cited 2010 January 12];

Available from: http://www.mos.org/nctl/our_nations_challenge.php. 11. Katehi, L., G. Pearson, and M. Feder, eds. Engineering in K-12 Education. 2009, The National Academies

Press: Washington, D.C.

12. Vande Zande, R. (2007) Design education as community outreach and interdisciplinary study. Journal for

Learning through the Arts, 3(1), 1–22.

13. Staw, B. (2006) Individualistic culture trumps teamwork. University of California at Berkeley (online).

Retrieved from www.haas.berkeley.edu/news/20060717_staw.html

14. Goldman, S. (2002). “Instructional Design: Learning through Design.” In J. Guthrie, (Ed.), Encyclopedia of

Education. Second Edition. New York: Macmillan Reference USA. 1163-1169.

15. Kolodner, J. L., Camp, P. J., Crismond, D., Fasse, B., Gray, J., Holbrook, J. & Ryan, M. (2003) Promoting

deep science learning through case-based reasoning: rituals and practices in learning by design classrooms,

in N. M. Seel [Ed.] Instructional Design: International Perspectives. Mahwah, NJ: Lawrence Erlbaum

Associates, pp. 89-114. 16. Reilly, Rebecca L., "Developing a Marketing Plan Using Design Thinking" (2011). Creative Studies

Graduate Student Master's Projects.

17. Potratz, Jennifer Lynn, "Human Centered Design Approach to Mentoring" (2012). Creative Studies

Graduate Student Master's Projects.

18. Brown, Tim (2009). Change by Design: How Design Thinking Transforms Organizations and Inspires

Innovation.

19. Bootcamp Bootleg D. School. 2010. http://dschool.typepad.com/files/bootcampbootleg2010v2slim-1.pdf.

Hassno Platner & Institute of Design at Standford University. 20. Simon, H. A. (1996). The Sciences of the Artificial, 3rd Edition, Cambridge, Mass: MIT Press,

21. Krippendorff, K. (2006). The semantic turn: A new foundation for design. Boca Raton, FL: CRC Press

Taylor & Francis Group. 22. Brown, Tim (2008). Design Thinking. Harvard Business Review, 86(6), 84-92.

23. Vand, Zande, R. Design Education as Community Outreach and Interdisciplinary Study. Journal for

Learning through the Arts, 2007. 3(10; p. 1-22.

24. Staw, B. Individualistic Culture Trumps Teamwork, 2006.

www.haas.berkeley.edu/news/20060717.staw.html.

25. Evans, D.L., McNeill, B.W., and Beakley, G.C., “Design in Engineering Education: Past Views of Future

Directions,” Journal of Engineering Education, Vol. 79, No. 4, 1990, pp. 517–522.

26. IDEO (2010). Human-Centered Design Toolkit 2nd Edition. Online:

http://www.ideo.com/images/uploads/work/casestudies/

pdfs/IDEO_HCD_ToolKit_Complete_for_Download.pdf.

27. Zhang, T., & Dong, H. (2009). Human-centred design: an emergent conceptual model.

Page 24.146.21

21

28. National Academy of Engineering (2005). Educating the Engineer of 2020. Washington, D.C.: National

Academies Press

29. National Academy of Engineering (2008). Grand Challenges for Engineering. Washington, D.C.: National

Academies Press.)

30. Huff, J. L., Patten, T. A., Wells, R. L., & Cox, M. Ansanm: A Cross-Cultural, Interdisciplinary Approach

to Learning Human-Centered Design. 31. Zoltowski, C. B., Oakes, W. C., & Cardella, M. E. (2012). Students' Ways of Experiencing

Human‐Centered Design. Journal of Engineering Education,101(1), 28-59.

32. Bringle, R. G., & Hatcher, J. A. (1996). Implementing service learning in higher education. The Journal of

Higher Education, 221-239. 33. Hatcher, J. A., & Bringle, R. G. (1997). Reflection: Bridging the gap between service and learning. College

teaching, 45(4), 153-158.

34. Lisman, C. D. (1998). Toward a civil society: Civic literacy and service learning. ABC-CLIO. 35. Astin, A. W., Vogelgesang, L. J., Ikeda, E. K., & Yee, J. A. (2000). How service learning affects students.

Los Angeles: Higher Education Research Institute, University of California.

36. Duffy, J., Barington, L., Moeller, W., Barry, C., Kazmer, D., West, C., & Crespo, V. (2008). Service-

learning projects in core undergraduate engineering courses. International Journal for Service Learning in

Engineering, Humanitarian Engineering and Social Entrepreneurship, 3(2).

37. Tsang, E., Marlin, D. and Decker, R. (1997). ‘Service-learning as a strategy for engineering education for

the 21st century.” Proc., 1997 ASEE Annual Conf., American Society for Engineering Education,

Washington, D.C. 38. Maguire, M. (2001). Methods to support human-centered design. International Journal of Human-Computer

Studies, 55 (3), 587-634.

39. ALLISON, G., CATTERALL, B., DOWD, M., GALER, M., MAGUIRE, M. & TAYLOR, B. (1992).

Chapter 2: Human factors tools for designers of IT products. In M. GALER, S.HARKER & J.

ZIEGLER Eds. Methods and ¹ools in ;ser-centered Design for I¹ (Human Factors in Information

Technology series, no. 9). Amsterdam: North-Holland.

40. Becker, K., Mentzer, N., & Park, K. (2012). High School Student Engineering Design Thinking and

Performance. Paper presented at the ASEE 2012 Annual Conference and Exposition, San Diego:

California.

41. Atman, C. J., Adams, R. S., Cardella, M. E., Turns, J., Mosborg, S., & Saleem, J. (2007). Engineering

design processes: A comparison of students and expert practitioners. Journal of Engineering

Education, 96(4), 359-379. 42. Atman, C. J., Chimka, J. R., Bursic, K. M., & Nachtmann, H. L. (1999). A comparison of freshman and

senior engineering design processes. Design Studies, 20(2), 131-152.

43. Atman, C. J., Kilgore, D., & McKenna, A. (2008). Characterizing Design Learning: A Mixed‐Methods

Study of Engineering Designers' Use of Language.Journal of Engineering Education, 97(3), 309-326.

44. Christiaans, H. H. C. M., & Dorst, K. H. (1992). Cognitive models in industrial design engineering:a

protocol study. Design theory and methodology, 42, 131-140.

45. Ennis Jr, C. W., & Gyeszly, S. W. (1991). Protocol analysis of the engineering systems design

process. Research in Engineering Design, 3(1), 15-22.

46. Guindon, R. (1990). Knowledge exploited by experts during software system design. International Journal

of Man-Machine Studies, 33(3), 279-304. 47. James, C. M., Goldman, S. R., & Vandermolen, H. (1994). Individual difierences in planning-related

activities for simple digital circuit design (Tech. Rep. No. 94-1). Nashville, TN: Vanderbilt University.

48. Rowland, G. (1992). What do instructional designers actually do? An investigation of expert practice.

Performance Improvement Quarterly, 5,65-86.

49. Sutcliffe, A. G., & Maiden, N. A. M. (1992). Analysing the novice analyst: cognitive models in software

engineering. International Journal of Man-Machine Studies, 36(5), 719-740.

50. Bursic, K. M., & Atman, C. J. (1997). Information gathering: a critical step for quality in the design

process. Quality Management Journal, 4(4).

51. Ericsson, K. A., & Simon, H. A. (1984). 1993. Protocol analysis: Verbal reports as data.

52. Mosborg, S., Cardella, M., Saleem, J., Atman, C., Adams, R. S., & Turns, J. (2006). Engineering design

expertise study, CELT Technical Report CELT-06-01. Seattle: University of Washington.

Page 24.146.22

22

53. Moore, R. C., Goltsman, S. M., & Iacofano, D. S. (1997). Play for all guidelines: Planning, design and

management of outdoor play settings for all children. MIG Communications, 800 Hearst Ave., Berkeley,

CA 94710.

54. Mawson, B., Beyond `The Design Process': An alternative pedagogy for technology education.

International Journal of Technology and Design Education, 2003. 13(2): p. 117-128.

55. Casey, S. (1993). Set phasers on stun and other true tales of design, technology, and human error. Santa Barbara, CA: Aegean Publishing Company.

56. Damodaran, L. (1996). User involvement in the system design process - a practical guide for

users. Behaviour & Information Technology, 15(6), 363–377.

57. Norman, D. A. (1988). The design of everyday things. New York, NY: Basic Books.

Page 24.146.23