Critique: Assessment and the Production of Learning

Critique: Assessment and the Production of

Learning

ELISABETH SOEP

University of California, Berkeley

Education researchers and classroom teachers have argued that the constant pressureto measure and rank students makes it difficult to shape assessment as an episode oflearning. Yet we know little about how learning moves in and through assessment ofany kind. Building on two national multisited studies, the research reported here usesethnographic techniques to examine learning within critique. Critique is a form ofassessment through which young people jointly judge their own work and that of theirpeers. The article focuses on episodes of critique within two nonschool sites for col-laborative production involving ethnically and economically diverse groups ofyouth—a community-based video project and an organization in which young peoplecreate radio stories for local and national broadcast. Learning environments such asthese draw voluntary youth participation and are organized around sustained projectsreleased to outside audiences. Findings indicate that critique manifests itself as anepisode of learning by engaging young people in joint assessment events that areimprovisational, reciprocal, and oriented toward the future of the work under review.Intra- and cross-site comparisons suggest that critique is likely to arise within specificconditions: when stakes are intense, metastandards are subjected to review, account-ability is mutual and interactively sustained, and interdisciplinary practice is man-datory. The article reviews various ways to conceptualize learning and argues in theend for a theory of learning as production, a way of making. Implications includenew ideas for research methodologies and new understandings of youth-adult col-laborations in learning and production.

Inside any given classroom within the United States, disparate forces, in-cluding state standards, federal policies, legally mandated procedures, andcommercial products, shape even tiny moments of student evaluation.Criticisms of classroom-based evaluation conventions are well established inthe education literature. Too much emphasis centers on measurement andranking and too little on how assessment can serve as an episode of learningthat is consistent with the larger pedagogical goals and practices of a givenclassroom, while also attuned to the needs and dispositions of individualchildren and youth.

Teachers College Record Volume 108, Number 4, April 2006, pp. 748–777Copyright r by Teachers College, Columbia University0161-4681

Despite these criticisms, the current sense of urgency around school ac-countability makes it very difficult for even the most sensitive educators tomodulate how they evaluate their own students. Standardized testing pro-cedures privilege discrete bits of knowledge over performances that revealactive modes of knowledgeability. Although theorists may conceptualizelearning as inquiry or a process of situated participation, arguing that mindsdevelop through active engagement with meaningful tasks in specific con-texts, rarely do mandated high-stakes evaluations focus on actual partici-pation in anything other than a testing situation. Many reforms talk ofassessment as instruction, but the fact remains that we know very little abouthow learning moves in and through assessment of any kind. The researchreported here examines learning within critique, a form of assessmentthrough which young people jointly judge their own work and that of theirpeers as they prepare that work for public release.

ASSESSMENT AS A MOMENT-TO-MOMENT ACTIVITY

Part of the difficulty in assessment reform stems from the fact that complexand collaborative learning tasks do not lend themselves to straightforward,easily replicable evaluation methods; such tasks require the production ofnew knowledge and experimentation with received ideas. It is not easy totrack and compare student learning when young people work together oncreative problem-solving projects. Evaluation under these conditions tran-spires not only through external appraisal but also, crucially, by way ofongoing self- and peer-assessment by participants in the activity itself. Per-haps we should be looking, therefore, inside learning events, to the dis-course of young people themselves, as we develop new strategies forassessment. Support for this line of argument is revealed in research cen-tered on assessment as a moment-to-moment collaborative affair. For ex-ample, in a recent study of young people involved in group workassignments, Cohen and her colleagues (2002) found that the more stu-dents evaluated their own work as it developed, using criteria presented atthe beginning of the task, the more motivated and task focused was theirdiscourse, resulting in higher quality products and more sophisticatedwritten reflections.

Empirical evidence supporting the value of self- and peer-assessment iscritical in efforts to reimagine the role of evaluation in education. Statedsimply, it appears that the act of assessment itself is too often not sufficientlyparticipatory. Participatory evaluation would regularly place young peoplein situations in which they must evaluate their own work and that of theirpeers to move the work toward completion and ultimate release. Researchon how assessment messages circulate in everyday discourse beyond

Critique: Assessment and the Production of Learning 749

classrooms advances this view. In traditional classroom formats, teacherstend to control assessment, with students playing at best a passive role(Cazden, 2001; Filer, 2000). In everyday conversation, by contrast,assessment is often deeply participatory (Goodwin & Goodwin, 1992;Linde, 1997). When one speaker utters an assessment in the course ofconversation, subsequent speakers tend to position themselves with respectto the appraisal that they have just heard, by conveying agreement or ex-pressing dissent.1 This process can be highly energetic, evident in flurries ofoverlapping, densely populated stretches of speech, as in a phenomenonespecially conducive to collaborative problem-solving that Tannock (1998)identified as ‘‘swarming.’’ In this sense, self- and peer-assessment can inviteheightened participation and represent a real educational opportunity toengage students more fully in their own learning.

Young people themselves, when faced with certain kinds of learningtasks—specifically, those aimed toward the release of original, collabora-tively produced material to an outside audience—are constantly developingthe very criteria that they will then apply to their work as they evaluate itsmerits. This activity can be described as a process of critique—the jointassessment of original objects and performances by producers themselves.

CRITIQUE IN THE ARTS

Models for critique exist in every field: Scientists evaluate one another’stheories as they prepare for meetings and clinical trials, just as architectsformally review design plans before pitching accounts, athletes screen gametapes to recognize and learn from the lessons of past performance, andpolitical teams endlessly debate the nuances of single words when compos-ing speeches and slogans. However, conventions for critique derived fromthe arts are especially relevant, given the highly established, if underrec-ognized, rituals for self- and peer-assessment that occupy a prominent placein arts education. Nonschool spaces for arts production in particular revealhow young people use critique in settings that draw voluntary youth par-ticipation—contexts organized around sustained creative projects culmi-nating in public release of young people’s work. A moment of critique inthese kinds of settings might involve a group of young people who areproducing a video and stop in the middle of a shoot to assess and radicallyrevise the choreography for a scene that is just not working as it was orig-inally imagined. Or picture a group of teen radio producers preparing aspecial series on youth, money, and marketing for a national show. They arediscussing the subtleties and controversies embedded within a spoken-wordpoem written by a 17-year-old as they consider the poem as a possibleelement in that series.

750 Teachers College Record

These specific moments of critique took place within two different sitesfor collaborative arts production: a youth video project called Cutaway, inwhich young people produce movies screened in cinemas and museumspaces, and a nonprofit media program called Youth Radio, in which youngpeople collaborate with adult producers on stories for local and nationalbroadcast.2 Representative episodes of critique in these settings containbroader implications for the design of participatory, assessment-rich learn-ing environments. Analysis of these interactive moments begins with adefinition of assessment as a process of seeing and responding to a givenpiece of work, and critique as a practice of mutual assessment that youngpeople use when they turn to one another as resources for judgment. Aftera brief review of literature on formal peer critique conventions in arts ed-ucation, and a clarification of the conception of learning being applied andadvanced here, the analysis is organized around four questions:

1. How does critique manifest itself in the process of moment-to-moment interaction? This question centers on the ways in which critiqueprovides access to specific learning opportunities by giving young peopleresponsibility for developing and applying judgment.

2. What are the implications of critique for learning practice? This ques-tion draws attention to the conditions that promote critique.

3. What are the implications of critique for learning theory? Response tothis question suggests a conception of learning as production.

4. What are the implications of critique for methodologies of research onlearning? This final line of inquiry suggests that models for critiqueamong youth contain lessons for the conduct of participatory, collabo-rative research.

RESEARCH ON ASSIGNED PEER ASSESSMENT

There is a strong, though underrecognized, tradition within visual arts ed-ucation, especially on a postsecondary level, of including learners in theassessment of their own projects and those of their peers through some-thing called a crit.3 A crit is a period of time, usually several hours, set asideduring an art class when students display work and discuss with instructorsand one another the strengths and weaknesses of each piece. Any mentionof crits to a group of practicing artists is likely to invite a spirited response.When told about my interest in crits as sites of learning and assessment, onepainter recalled his professor hurling a beer can at his painting during acrit. Another watched in dismay as an instructor tossed student paintingsout of a high-rise building window. I have had my own difficult, although


admittedly less dramatic, experiences in crits—for example, waiting mis-erably until my monoprint was picked last in a crit in which students de-cided in what order to consider individual pieces of work. It felt like beingthe last kid chosen for the kickball team in gym class.

Crits evoke other, highly positive narratives as well. Research based inpostsecondary settings reveals that many artists credit crits for motivatingtheir productivity by introducing intense deadlines, exposing them to newtechniques and interpretations, forcing them to refine their habits of per-ception as related to production, and building their fluency in vocabulariesof art (Soep, 2000). Unlike criticism, in which outsiders to an art-makingexperience describe and evaluate a given piece, in crits, artists themselvesmake judgments about their own evolving work, and they find language todescribe what they see, like, and dislike in the efforts of others who arelearning alongside them.

There is relatively little attention, however, to crits in educationscholarship (Bulka, 1996). The research that does appear tends to be in-strumental, highlighting features of effective peer critique. These studiesoften focus on university settings, calling for crits marked by a number ofauspicious characteristics, including discussion driven by aspirationrather than ideological position (Rogers, 1996); close focus on specificqualities of the work under review (Roth 1999); a climate of ‘‘spontaneity,empathy, and equality’’ (James, 1996, p. 153); and a commitment to‘‘provisionalism’’ in commentary offered in critique (Wernik, 1985). Whenthese qualities are not in place, research suggests that crits fail to informartists’ practices and judgments in any meaningful way, while making par-ticipants feel as if they are ‘‘being hung naked on the wall and ridiculed’’(Wernik, p. 194).

Even fewer researchers have published refereed studies of critique inK–12 visual arts classrooms. Barrett (1988, 1994, 1997, 2000) is perhaps themost notable exception (also relevant are Blythe, Allen, & Schieffelin Po-well, 1999; Cotton, 1981; Ende-Saxe, 1990; House, 2001). Barrett said thatcrits among young artists should have a purpose, invite participation, in-clude positive and negative reasoned judgments, emphasize interpretationover evaluation, and privilege the viewer as the most important participant.Facilitators of crits should be prepared, said Barrett, to ask and elicit goodquestions and to address fundamental issues pertaining to intent, content,subject matter, form, relationship of media to materials, sources of artisticinfluence, and social issues. Barrett’s work and that of others in the researchliterature provide a useful conceptual framework and a practical set ofprinciples for educators who facilitate crits in their classrooms.4 The exist-ing literature also exposes some gaps in our understanding of the processesof self- and peer-assessment as a property of learning. Specifically, weknow very little about how critique materializes spontaneously within


collaborative undertakings when it is neither scheduled, nor staged, norassigned by anyone but young people themselves out of some kind of tacitrecognition that joint assessment is in order.

CONCEPTIONS OF LEARNING THAT INFORM AND ARISEFROM CRITIQUE

Moving from assigned crits to emergent critique entails a shift in focus frompreplanned feedback sessions to moments of judgment brought on by thedemands of collaborative production. This shift requires a working concep-tion of learning, or, more specifically, what Cohen and Ainley (2000) called a‘‘theory of cultural learning’’ (p. 92). Such a theory accounts for how youngpeople study and develop ‘‘school knowledge’’ and how they ‘‘learn todance, or knit, or make love, or ride bicycles, or horses, or play football, orwrite graffiti or poetry, or tell jokes, or tall stories, practice safe sex, usecomputers, play musical instruments, conduct experiments’’ (p. 92).5

One well-established tradition for theorizing learning is by way of ac-quisition models, which frame learning as the internalization of discreteinformation that is then transferred to new contexts. Students memorizedates, study rules and patterns, and master terminology and habits asso-ciated with a given task or field, and then demonstrate those achievementsthrough systematic examination. Although this view may be intuitively use-ful, contemporary learning theorists have raised concerns about acquisitionmodels, arguing that they make learning passive and equate knowledge toan exchangeable commodity. Nevertheless, it has been argued that the ideathat students acquire and eventually transfer information continues todominate the assessment literature, at least implicitly, to the extent thatresearchers do not always fully develop alternative theories of learning thatdrive their analyses of assessment conventions and proposals for change(Delandshere, 2002).

On the opposite end of a theoretical spectrum from acquisition models isa view that learning takes place simply by virtue of being alive and adaptingand adjusting to new conditions. Learning from this perspective is aneveryday affair—a way of handling the world and what happens within it. Itis less a bounded phenomenon that can be neatly isolated and tested than itis a natural product of being in the world, taking in new information andresponding to it, all the while registering the ‘‘lessons’’ that arise out of livedexperience. This view is useful to the extent that it frames learning as anordinary practice, providing an important corrective to the tendencyto locate learning exclusively within settings where obvious and deliberateinstruction is taking place. Perhaps this conception is therefore a necess-ary step in opening researchers’ eyes to the existence and significance of


learning environments within young people’s families, peer groups, neigh-borhood streets, and media consumption practices, and in school class-rooms. This view is, however, so vague and far reaching as to have limitedanalytical utility.

A third view reconciles an interest in learning as a process of acquiringinformation, with an interest in learning as a property of everyday expe-rience, while at the same time offering ways to account for the creation ofnew knowledge. This third view says that learning is a process of situatedparticipation, in which new information enables people to reorganize theirrelationships with one another and with respect to their joint projects. Inother words, learning means shifting the conditions that shape minds, cir-cumstances, actions, and products. In this situated view, knowledge is co-constructed within communities of practice (Chaiklin & Lave, 1996;Greeno, 1998; Lave & Wenger, 1991; Papert, 1991; Rogoff, 1994; Varenne& McDermott, 1999; for recent critical discussions, see Anderson, Reder, &Simon, 2000; Fox, 2001; Hacking, 2000; Sfard, 1998; Young, 2000). Thisnotion of learning is a useful starting point, although this view will not bewhere the analysis ends. Study of critique pushes this situated participationmodel in a new direction—toward a view of learning as production, a way ofmaking. Conceiving of learning as production provides new theoreticaltools to help us work through some of the questions that the situated par-ticipation perspective leaves open—specifically, how learning migrates fromone situation to another or, more to the point, how a single situation canitself contain multiple imagined contexts. Critique is a key process wherebylearners produce these imaginary contexts by conjuring hypotheticalscenes, responses, and consequences for the decisions made within theprocess of production. In this sense, as we will see, assessment can do morethan measure learning. It can serve as the fuel, so to speak, that moveslearning forward beyond a single situated moment in time.

SITES AND SCENES OF CRITIQUE

This argument for a particular way of theorizing learning surfaces fromscenes of interaction within two nonschool sites for creative media produc-tion, Cutaway and Youth Radio. A focus on settings such as these is con-sistent with a growing interest within the field of education in learning,assessment, and youth development beyond school walls (cf., Campbell,2001; Heath & McLaughlin, 1993; Hogan, 2002; Hull & Schultz, 2001,2002; Mahiri, 2003; Noam, Biancarosa, & Dechausay, 2002; Paley, 1995;Soep, 2002). Research in this area acknowledges that young people spendonly a relatively small percentage of their time inside classrooms and thatthe best community-based projects seem able to draw active, committed,


and highly impressive performances of learning among youth. Languagedevelopment appears to extend well into later childhood and adolescencethrough extended involvement with community-based projects over sus-tained periods (Heath, 1998b).

These observations do not, of course, mean that schools matter less,only that life beyond school has the potential to reveal new insights relevantto the design of learning environments likely to draw youth engagementand high quality performance. The same students who might strugglein a remedial English class write and perform spoken-word poems thatbring critical audiences to their feet. Young people with sporadic attendancein classrooms or those who have dropped out of school entirely willconsistently attend nonschool projects such as media programs, theatercollectives, or aerosol art crews, moving into leadership positions andmaking long-term commitments, advancing their own skill levels whilepushing the arts that they practice in new directions. Evidence of youthengagement and achievement along these lines is reported in two nationalstudies that greatly inform the present analysis, through my own partic-ipation in those studies as a researcher and ethnographer. The firstis Harvard University’s Project Co-Arts, a study of pedagogy, assessment,and neighborhood-based development within community arts centers inlow-income settings (Davis, 1998; Davis, Soep, Maira, Remba, & Putnoi1993). The second is a Stanford University-based study (1987–1997)6 ofyouth learning and leadership in community-based organizations, which in1995 began to focus on arts-based sites, given the especially notable contextsand outcomes for learning that these settings appeared to provide (Heath,1993, 2001; Heath & Ball, 1993; Heath & Soep, 1998; Soep, 1996, 2000,2002, 2004, in press).

The organizational and interactive features linked to effectivecommunity-based youth organizations are well documented in the existingliterature—for example, opportunities for intensifying youth participationand leadership, and sustained projects organized around cycles of planning,preparation, practice, and performance (Heath & McLaughlin, 1994a,1994b). Some investigators urge schoolteachers to forge relationships withcommunity projects, suggesting that classroom-based educators mightprofitably model some aspects of the kinds of learning environments thattake hold more commonly beyond school walls. Others point to the con-siderable structural barriers that make these kinds of relationships verydifficult, given the compulsory, highly regulated character of education inschool. Overall, it seems important, as we pay more and more attention tocommunity-based learning environments for youth, not to imply that animpenetrable boundary separates them from schools (Hull & Schultz,2002). Such a boundary does not always exist for young people themselves,and innovative educators in both realms of practice very often share a


commitment, and also the capacity, to create conditions in which youthserve as producers and judges of their own development (Weis & Fine,2000).

Moreover, researchers are increasingly reckoning with the dangers ofromanticizing nonschool learning environments as some kind of educationalpanacea. It is all too easy to portray these places as ‘‘free’’ spaces whereyoung people can escape (or, more problematically, be rescued from) theforces of inequality that they face in other institutional settings, includingschool (Hull & Schultz, 2002). Hogan (2002) identified the ways in whichpower dynamics can undermine community-based learning by shuttingyouth out of meaningful involvement, limiting their participation to menialtasks and potentially doing more harm than good, as yet another example ofan experience whose reality looks quite different from the ways in which itwas initially presented to youth participants. Trend (1997) considered arts-based community projects in particular, pointing to the striking unevennessamong such projects in terms of product quality and the depth of youthengagement; his analysis suggests that young people are often invited into aproject to provide autobiographical content and access, but they may havelittle say over how, where, and to whom the work is actually presented. Inher study of a youth media project, Fleetwood (2005) sharpened and ex-tended this point about the limits to youth involvement in community-basedlearning, and she cautioned that such projects have a tendency to pursue thefantasy of an authentic youth voice, which itself often embodies a sensa-tionalized portrayal of racialized ‘‘urban’’ youth experience.

What is striking in the literature highlighting the positive potential andthe pitfalls of nonschool learning environments is the extent to which as-sessment seems to be a kind of pivot point. Effective community-basedprojects engage young people in the process of judging their own work andthat of their peers, and indeed this level of participation in assessment is ahallmark of high-quality sites (Heath & Soep, 1998). At the same time, manyof the problems that can arise with these projects are revealed in the ways inwhich young people are excluded from the process of formulating judg-ments and acting on them—when they are called upon, for example, to telltheir stories in graphic detail but then dismissed when it comes time tomake editorial decisions and transform confessional material into refinedworks. It is therefore especially relevant, given these current debates ineducational practice and research, to look closely at the assessment practicesthat take place within such projects, to illuminate how learning happensthere and how learning can be theorized in a broader sense. It is importantto recognize that critique finds no easy path into school processes andproducts, for the institution is not set up to take assessment from studentsinto account within high-stakes measurements such as standardized tests.Nevertheless, classroom educators who practice performance-based


approaches to assessment within the everyday flow of their teaching regardevaluation as an episode of learning. They create ‘‘real-world’’ events inwhich young people perform ‘‘mastery’’ of a given subject for outside au-diences and panels, and in preparing for these performances, students of-ten find themselves in a position to critique their own work and that of theirpeers (Darling-Hammond, Ancess, & Falk, 1995). Daily assessment insideschools is a social practice in which students can be active agents (Filer,2000); critique is therefore a powerful resource for teachers who see youngpeople as partners in the production of learning.

The research on nonschool learning environments reported in this ar-ticle has evolved through two phases. Building on the national studies notedpreviously, the first phase, from 1991 to 2000, included study of youthlearning within more than 20 community-based organizations, including ateen drum corps, an improvisational theater troupe, a mural and graphicarts project, a zine-writing group, and a video production project (Cutaway)in cities in New England, the southeastern United States, and on the WestCoast. These last two sites were studied in particular depth for comparativepurposes, including, in the case of Cutaway, field work over 1 year thatentailed more than 190 hours of intensive data collection and tape record-ing to yield a language corpus for discourse analysis.7 This phase of theresearch, although centered on projects facilitated by adult leaders, focusedespecially on moments of critique among youth as they formulated judg-ments of their own work and that of their peers. My own participation as aresearcher in these moments of critique was minimal because it would haveseemed problematic, given the nature of the research, for my own habits ofcritique to shape the phenomenon central to the study. In the second phaseof the research, from 2000 to the present, this emphasis on observation withlittle participation gave way to a different arrangement entirely. Phase two isan action-research project within one youth media organization, Youth Ra-dio, where I work actively with young people to produce radio stories forlocal and national broadcast. My own voice is, therefore, one among manyin the exchange of critique. This phase of the research looks more closely atthe workings of critique between young people and adults who are mu-tually, if asymmetrically, engaged in a project for which both parties will bejudged once the broadcast airs to significant audiences.

The analysis offered here focuses on one episode of critique from eachphase of the research, the first based at Cutaway and the second at YouthRadio. These episodes were selected for their representativeness; thesewere not random, nor were they exceptional occurrences. There were, tobe sure, long stretches of time within both research sites when no critiqueamong youth could be heard: when instructors lectured on a particulartopic or demonstrated a new piece of equipment, or when young peopleworked silently in a studio or edit suite. Episodes of critique such as the ones


that follow predictably arose when the process of production itself requireda moment of display or performance—when a young video maker had topresent her script to the actors who would be playing the parts, or when aradio producer was recording his narration in an announce booth and thepeer who was engineering that session provided feedback on voicing tech-nique. These ‘‘naturally occurring’’ assessment junctures took place overthe entire course of a production project, from the earliest stages of ideageneration, through the final hours before the work would be released, tothe aftermath of an exhibition or broadcast. It grew especially intense whensome kind of conflict arose—when participants needed to make a crucialdecision—and there was disagreement as to the best way to proceed. Withinthese kinds of moments, critique made its strongest appearance as a re-source for identifying and solving problems through a process of assess-ment, in this sense significantly repositioning assessment as a practice thatfuels, rather than merely measures, learning. It is important to clarify,however, that conflict is defined not necessarily as an acrimonious encoun-ter, but instead as a moment when young people face some obstacle inmoving their work forward and when there is no immediate consensus onhow to proceed, nor is there an adult who single-handedly can make thatjudgment on their behalf.

SCENARIO 1: CRITIQUE AMONG YOUTH AT CUTAWAY

Founded in 1995, Cutaway is located in a major West Coast city and involvesprimarily working- and middle-class youth of color and led by a White artistfrom a working-class family.8 In the specific Cutaway project highlightedhere, 8 young artists ranging in age from 14 to 18 spent one summer andfall producing an original movie, acting and serving as crew members onone another’s productions. Their individual videos were ultimately pro-jected simultaneously in a video installation, meaning that they had to makesense as standalone narratives and also as part of a total visual environmentin the gallery where the work premiered. While learning the basic technicalskills of video production, the young people spent several weeks brain-storming ideas for their individual movies and narrative and visual themesthat would resonate across all the work, lending coherence to the overallinstallation. Lila, the adult artist in charge, launched the project with a loosesuggestion that perhaps the group might want to do movies that related insome way to their neighborhoods, but by the time the young people haddeveloped scripts and begun shooting, they had diverged considerablyfrom that initial point of departure. There was one mockumentary aboutflirtation rituals between boys and girls, an experimental documentary onpublic art, and an earnest, soap opera-like love story about romance acrossrace and class, to name a few. Every phase of the process that led up to these


completed movies was punctuated by episodes of critique in which theyoung people reviewed one another’s works, discussing and debatingstrengths, weaknesses, and possibilities for change.

One particularly heated instance of spontaneous critique arose during amiddle-of-the-night shoot, when Simon, a 17-year-old public high schoolsenior, was directing and starring in his own fictional movie about hostilitybetween two generations of Asian American youth. It was after midnight.The group gathered in an urban alley, running extension cords out ofadjacent apartments, rigging lights, mics, and camera. Everyone in thegroup was exhausted and frustrated. The day had started at 9:00 a.m., andmore than 15 hours later, it became apparent to the full group that Simon’sfight scene was falling apart. It reached a point where Lila, the adult artist,‘‘opted out,’’ as she said. She told the youth cast and crew that they wereon their own to figure out how to rework the scene and started packingup the set.

Within this tense environment, it came time to shoot the pivotal momentin Simon’s piece: when his character gets into a physical struggle with an-other character, played by Chanpory, at which point a gun accidentally fires,killing Simon’s character’s brother (played by Simon’s real-life brother,Bryant). While all this action was happening on camera, a great deal of workwas also taking place behind the scenes. Cleo scrutinized the action on afield monitor, a portable television that allows video makers to see whattheir footage looks like on screen. Claire operated the camera. Lydia, an-other member of the group, watched and waited. Claire called action.Simon and Chanpory lunged toward each other and began the struggle,with Claire rolling tape until Cleo stopped everything abruptly, saying ‘‘I’mtelling you, Simon, it’s not working. This is not working.’’ Simon at thatpoint was still focused on Chanpory, telling him exactly how to pull his armto make him fall, but Cleo was insistent, calling Simon’s name repeatedlyuntil he finally turned to her, even as Lydia, in the background, mutteredunder her breath, ‘‘I feel like screaming.’’ Finally with Simon’s attention,Cleo began to explain what she had seen. ‘‘All right, when I was looking atit, it looked really weird, because—okay, what do you want the people tosee? You wanna see all four of you guys, or just two?’’ Simon and Chanporyimmediately began answering simultaneously, and even after Cleo toldChanpory to let Simon talk (it was his movie after all), Chanpory continuedto direct the scene from his own point of view. ‘‘No! I don’t want to hearfrom you!’’ she said, putting her hand up over Chanpory’s face, saying ‘‘Idon’t even want to hear from you.’’ Chanpory laughed and backed off,letting a surprised-sounding, ‘‘Sorry!’’ pass through his lips. Simon thenexplained his original vision of the scene, a mix of medium shots and wideshots, from various angles. Cleo repeated his descriptions, making sure sheunderstood what he had in mind and prompting him to go on:9


1. Cleo: Okay, then what? That’s okay. We’ll do that on theclose up, so then, this is what, when you guys arefighting, you make sure Bryant is facing—

2. Bryant: [Yeah, I did. I turned him around.3. Cleo: [these people, because I only saw your back.4. Bryant: You have to get up though.5. Chanpory: Yeah, you guys have to find a way to get up.6. Cleo: And make it fast, okay?7. Bryant: All right, [after you get shot, lay down on the floor

away from the camera.8. Chanpory: [Okay. When you come after me, focus on this hand,

and just put both hands—9. Cleo: And you guys, you guys, you guys, you’re not fight-

ing. You’re acting. So let the other person take con-trol, if that has to be, you know what I mean. Don’t bestruggling, cause that looks stupid.

QUALITIES OF INTERACTION: ACCESS TO JUDGMENT

A fleeting exchange such as this might easily be missed by observers lookingfor obvious or tightly organized evaluation events (Lee, 2001). And yet thiskind of episode, as representative of a process that took various forms acrossthe production cycle of the Cutaway project, reveals how critique manifestsitself within moment-to-moment interaction, providing access to specificlearning opportunities by giving young people responsibility for developingand applying judgment.10

First, this is an episode of joint assessment, meaning that young peoplewere turning to one another to see the work unfolding before their eyes innew ways, evaluating its merits and pointing to its shortcomings. Critique isoften shot through with obvious evaluative terminology—what linguists callassessment tokens—in this case, ‘‘It’s not working,’’ ‘‘It looked funny,’’ ‘‘Thatlooks stupid,’’ or even ‘‘I feel like screaming.’’ Youth in critique can alsoconvey assessment in less explicit ways—by silence, for example, or throughinterpretations that imply respect for the work under review, or with theunspoken ratification of uptake. In making these kinds of moves at thisjuncture within Simon’s shoot, young people are constituting a circle ofdecision makers at the precise moment when the adult in charge has stra-tegically, if temporarily, vacated her position. The learning dynamic in placehere is fueled in part by the mutual understanding that ultimately the workmust be completed in a timely manner (before they are kicked out of thealley), and that the finished product will be viewed and judged by a criticaloutside audience.


Particularly striking in the data both within Cutaway and across researchsites were instances in which young people dramatized narratives invokinganticipated audience members, and other imaginary interlocutors, enactinghow those hypothetical figures would respond to the work. They conjuredcharacters and then voiced their own assessments as if they came throughthat character’s mouth. In this excerpt, Cleo strategically alluded to whatSimon wanted ‘‘the people’’ to see, aligning her own questions with abroader audience view. In this sense, she amplified her voice, re-contextualizing the present decision—which was being made among peersand collaborators in an urban alley—by projecting the consequences of thatdecision onto an exhibition that would ultimately be assessed by a muchwider public. The young people themselves carried responsibility for judg-ing what they wanted those people, that projected audience, to see. It mustbe stated, however, that it was only after months of working together thatthe adult artist could leave the young people to resolve this crisis; they haddeveloped habits of critique and a shared understanding of one another’sprojects, which allowed them to persevere and see both weaknesses of thework and possibilities for salvaging Simon’s vision.

Second, critique entails moment-to-moment improvisation. Cleo, Simon,and the others did not have the luxury of planning out in advance how tomanage this particular microcrisis of representation. With little time toprepare what they would say, the young people improvised in the same waythat jazz musicians do on stage—by way of simultaneous composition andperformance, making ‘‘in-flight’’ decisions about problems without prefig-ured correct answers (Eisner, 1992; Thibault, 1997; see also Lee, 2001).They could also immediately sense how interlocutors were receiving theirfeedback and spontaneously adjust their commentary. Critique is thereforenot only iterative but responsive, evolving out of the specifics of a givengroup within a particular context. Enormous planning went into Simon’smovie, but not every detail was predictable. The young people necessarilyhad to improvise their work on the movie, in large part by exchanging andacting on impromptu assessments. This particular situation was a kind ofworst-case scenario—Simon’s carefully choreographed vision did not trans-late to the screen. Improvising one’s way through that kind of crisis withoutan absolute authority who could come in and save the day meant calibratingthe critique to the dynamics of that precise moment. Never before in theCutaway process had someone summarily stopped someone else from talk-ing (by inserting a hand in front of another person’s face) as Cleo did here.But she apparently sensed that the urgency of the present situation neces-sitated that move, and the ensuing interaction appears to suggest that shewas not mistaken. Along with this example operating on the level of in-teraction, there were also technical skills that allowed these young people, inthis moment, to improvise their way through this instance of critique; their


mastery of vocabulary such as wide shots and medium shots allowed themto speak in a kind of shorthand through which they could express, evaluate,and finally execute the scene. Critique, in a broader sense, entails impro-vising a course of action out of the often conflicting positions circulatingwithin the group. Youth work toward a standard that emerges from theircollaboration with a professional and their developing knowledge of modelsof ‘‘good work.’’

Reciprocity is a third defining feature of critique as an opportunity forlearning through the exercise of judgment. Within this moment duringSimon’s shoot, the young people exchanged perspectives, identified prob-lems, and proposed solutions, not always harmoniously. Cleo edged into adirectorial role, even in a movie technically directed by one of her peers.Once this episode passed, however, the dynamics of reciprocity shifted.Later this same night, the group began shooting a scene from Cleo’s movie,and soon they were critiquing her work. They energetically debated wheth-er it would be clear that her cop character, who spouted antiyouth rhetoric,was not expressing Cleo’s own views, but exposing the perspective that shewanted to condemn.

Reciprocity is what distinguishes critique from one-way structures ofevaluation: Someone from outside an experience passes judgment on thework without that person’s own efforts also being subjected to feedbackfrom the group. Reciprocity also differentiates critique from criticism, in theliterary or aesthetic sense; reviewers compose evaluations of completedwork without ever actually having to come face to face with author or artist,who may or may not even still be alive. All participants in critique subjectwork to joint review; there is a mutuality of engagement and vulnerability tothe judgment of others.

Fourth and finally, in critique there is an orientation toward emergingwork. Although participants in critique often (sometimes to a problematicextent) look back toward the intentions that drive certain creative decisions,they consider where the work comes from to determine where it needs togo. Within this scene from Simon’s shoot, the young people repeatedlyprojected a sense of future for the work and for their decisions even as theyentertained ‘‘past tense’’ considerations—including what Simon had orig-inally envisioned for the piece and how specific actions registered on screen.The youth projected what they could and could not do, forecasted howcertain moves would look, and asked questions to clarify planning for thenext take. By projecting an imaginary sense of future for the work, youngpeople in critique edge toward actual completion of a given project whilealso developing a broader sense of how that which they have seen andlearned in one project might apply to new endeavors in related fields.

The imprint of critique can be seen on finished products through theadjustments that authors have actually made on the basis of peer feedback,


and evidence in some cases that authors chose not to bend to every judg-ment offered by colleagues and collaborators. In the case of Simon’s movie,the change that came over the work as a result of critique was evident in thecontrast between the ways in which his peers initially reacted to his scenes,with considerable negative feedback and frustration, and the response thathe received at a rough-cut screening, just days before the final exhibit,attended by Cutaway youth and an adult professional editor consulting onthe project. When it came time to watch the fight scene and accidentalshooting, some in that audience were nearly speechless, others offeredemotional praise, and more than one person in that room actually tearedup, having been swept into the drama of Simon’s story playing out on thescreen.

SCENARIO 2: YOUTH AND ADULTS IN CRITIQUE AT YOUTH RADIO

The episode of critique drawn from Cutaway gives a picture of whatthe process can look like and what learning opportunities can open upwhen young people themselves manage the process of judgment. Thissecond example of critique shares the defining features identified above: Itis an instance of joint assessment, an improvisational, reciprocal processoriented toward the future of the work under review. However, in this case,the dynamics of critique include young people in a relationship with adults;both parties depend on each other, in obvious and subtle ways, to completework that meets professional standards. Episodes of critique involvingyoung people and adults take place almost daily at Youth Radio, whereadult producers such as myself collaborate with youth reporters on storiesfor local and national broadcast, most notably on National Public Radio.Young people come into the program from San Francisco Bay Area publicschools, and 80% are working-class youth and youth of color—not thevoices typically dominating public airwaves, particularly as reporters, com-mentators, cultural critics, and analysts. An ethnically and economicallydiverse staff brings diverse professional experiences to the work, includingbackgrounds in broadcast journalism and music programming, youthdevelopment, activism, and education.

The young people come to their first class at Youth Radio on a Wednes-day, and by Friday, they are on the air for their live show in which they DJmusic segments, write and deliver commentaries, announce the news, andproduce public service announcements. Young people who are especiallyinterested in audio narrative pursue advanced training and eventually ar-rive in Youth Radio’s newsroom, the primary source of the organization’snational stories. Youth Radio has aired national stories in the last few yearson topics including sexuality in middle school, standardized testing boycotts


in low-income districts, minors who volunteer as subjects in medical re-search, math requirements as a barrier to college retention, and commen-taries from Palestinian and Israeli youth living through the crisis in theMiddle East. Youth reporters meet weekly in editorial meetings, where theypitch story ideas and frame narratives around characters, ambient sound,archived media, and music. The process of actually producing a story in-volves gathering tape, writing a detailed script, recording narration, dig-itizing these materials, and using editing software to mix the piece. Youngpeople carry out each of these phases with hands-on guidance from adultproducers. Peer critique is built into a place like Youth Radio on multiplelevels—in editorial meetings, for example, or when two people go out to-gether to get tape for a story, or when one person is engineering anotherperson’s piece. There is also a longstanding model of peer teaching at YouthRadio, where young people serve as editors, voice coaches, and studiomentors to others just learning new skills. Within the apprenticing cultureof Youth Radio, these peer teachers are simply more experienced youthwho have been working for some time at Youth Radio, reaching the point atwhich they are prepared to educate others; in some cases, their students areolder than they are.

Recently, Youth Radio solicited a poem from 17-year-old Rafael SantiagoCasal as part of a series that the newsroom was producing for publicradio on youth, money, and marketing. Rafael writes and performswith a spoken-word poetry program called Youth Speaks, for whichI serve on the education committee of the board. Rafael delivers his poemin a head-spinning rush of words and images, peppered with sexualmetaphors and profanities (which are, at least to this listener, neithergratuitous nor predictable). This particular poem is a full-frontal attack onthe style industry, which he says uses child labor to commodify and sellyouth culture back to kids at a price that they cannot afford. Rafael startswith a mocking reference to ‘‘the man with fashion sense’’ who had thebright idea to tell kids what they need, creating ‘‘a million martyrs’’identically dressed in sagged jeans, major league team jerseys, silver chains,and $200 shoes. Later in the piece, Rafael moves from ‘‘the man’’ as thetarget of his condemnation to a corporation he genders as a ‘‘damn fine’’female who offers sex with just this catch: ‘‘you got to brand it andlavish and ravage your own/image after we ho ya and handle your ownevery dollar/until everyone marches to the same beat.’’ He links these mediamanipulations to a kind of ‘‘mental incarceration’’ not unrelated to theforces that ‘‘ended in the unfortunate/substantial transformation of ourpentagon into a quadrilateral.’’ And finally Rafael concludes, in this finalpassage, with an image of his own body controlled by the manipulations hedescribes—an ironic message, given the penetrating critique he has justproduced:


From BET to MTV we got PYTs11 marketing sex appeal the

all mighty breast appeal

It sells, yes but reflects poorly on the women ya feel

Got us thinking woman? Oh you mean woo-man here to woo

the man from the man to the man just to do the man lie

down dick ride and screw the man please she fuckin

rules the man this is just the plan they planted

embedded empowering them through the image in women

and men til corporate America owns my dick, they tell

it where to be what to do and how high to go so

frequently that i don’t own it anymore I feel lied to

We move to their currents like a tide pool

A million martyrs marching to the same beat

What an eye full

Public radio listeners will know that this is not the kind of narrative mostoften broadcast on those airwaves. Youth Radio producers were struck bythe power of the poet’s message and the intricacy and lyricism of his im-agery. We also had to consider the broadcast standards we would face inpitching the poem to our outlets’ editors, who sometimes traffic in morestraightforward reporting using established news formats. Through e-mail,we shared our feedback with Rafael. He would need to consider shorteningthe poem simply because of the slots allowed for ‘‘creative’’ segments suchas this one on a traditional public affairs show. Federal CommunicationCommission guidelines prohibit cursing and graphic sexuality, and we sug-gested that there were some passages whose subtlety might be hard tofollow for radio listeners on their afternoon commute.

In response to this e-mailed critique and suggestions for revision, Rafaelbasically declined to participate. Perhaps we had missed the message of thepoem, he said, which was about media manipulation of a personal truth.Was that not what we were asking him to do by requesting significant edits?The poet’s response set off an exchange of subsequent e-mails and face-to-face conversations, whereby he and we figured out a way to move forwardwith the pitch of his piece, given our editorial considerations and hisinsistence on the integrity of the message.


This episode begins to suggest the relationships between youth, adults,youth-serving institutions, and public cultures that enter and help shapecritique. What drew our interest was the content of Rafael’s poem, from avantage point of a listener, as a curator of youth voices. But Rafael, throughhis response to our critique, brought us inside his message, as he says—forcing reflexivity about the practice of critique, its functions and limits. Thelearning that arose here—for Rafael, and for the others involved in thisexchange—inserts adults inside the learning opportunities made possiblethrough critique. Young people and adults are jointly and reciprocally as-sessing one another, through moment-to-moment improvisation, alwayswith an eye toward a future for the unfolding work. The adults involved—their own habits, aims, and prospects—are very much embedded withinthese conditions, in any context in which the logics of youth developmentand professional production meet through the demands of mutual assess-ment. The version of the poem Rafael ultimately recorded for Youth Radio,which eventually aired on a weekly public affairs show, provides evidence ofthe impact of this episode of critique. In this case, however, unlike theCutaway example, it is the absence of change in the finished product that ismost striking and significant. The substance of the piece did not shift, andonly the profanities, which would have disqualified the poem from broad-cast on any network, were eliminated from the artist’s original.12 However,the learning that developed for the young people and adults who partic-ipated in these negotiations has lingered beyond this particular encounter,informing subsequent moments when it has come time to make judgmentsabout work that might push listeners to listen, and think, and see theirworlds in new ways.

IMPLICATIONS FOR LEARNING PRACTICE: CONDITIONSFOR CRITIQUE

Episodes of mutual assessment drawn from Cutaway and Youth Radioprovide a sense of the distinct learning opportunities made possiblethrough critique, as evident in the nuances of interaction and in the detailsof finished products. However, this kind of analysis cannot sidestep an im-portant set of questions: What guarantee do we have that critique will steeryouth down the right path? Specifically, would it not have been better tohave a professional choreographer on hand to monitor Simon’s fight sceneand simply place those bodies where they belonged? Is it not possible toargue that part of the frustration everyone felt during that midnight shootwas that everything at that moment was up for joint debate? Likewise, in thecase of Youth Radio, would it not it have been much more efficient had theyoung people and adults involved in the critique surrounding Rafael’s


poem simply taken that piece as it was from the outset, without getting intothe negotiations that in fact ensued? At its worst, cannot critique devolveinto something like a focus group, where every artistic decision passesthrough a consensus-generating process that ultimately compromises idi-osyncratic approaches and brave departures of imagination?13

The simple answer to these questions is yes. Critique can be misguiding,whether young people are alone in critique or formulating judgments withadults. The same can be said, of course, when adults are engaging in theirown joint creative pursuits. When scholars edit anthologies, or submit ar-ticles to refereed journals, or coteach classes, or plan conferences, it is al-ways the case that joint assessment might ultimately steer individuals andgroups in ill-fated directions or yield disparate and contradictory recom-mendations. And yet as young people learn to negotiate episodes of jointassessment, they develop habits and strategies for judging the quality oftheir own and their peers’ work. They need practice in the exercise ofjudgment if they are to realize that they need not rely exclusively on anoutside authority with the absolute power to evaluate their performanceseven if ultimately, other people’s assessments matter deeply to them andhelp determine the fate of their work.

There is also a second, and more complicated, answer to the question ofhow we know that critique will actually have a positive rather than negativeeffect on a given project or learning environment: to identify the conditionsassociated with critique as a necessary practice rather than contrived activity.Ironically, these conditions evoke and invert some of the principles mostcommonly associated with standardized testing in schools. Four conditionsemerged from my analysis across research sites where critique does anddoes not take hold. Comparison involved systematically reviewing tran-scripts and other records of interaction over the course of full productioncycles at the research sites to identify under what circumstances, withinspecific programs and across different organizational models, critique tend-ed to emerge among youth and between young people and adults. Critiqueerupts as a resource for learning when:

1. STAKES ARE INTENSE

This observation is counterintuitive in the sense that we tend to positionnonschool activities, especially those based in the so-called expressive arts,outside the academic subjects prioritized in school. English and math classesare what really matter, the logic goes, and the beauty of the arts, and ‘‘ex-tracurricular’’ environments in general, is precisely that they offer a kind ofoasis from the pressures that young people face in more rigorous schoolsubjects (Siegesmund, 2002; Soep, 2002). But in nonschool projects inwhich young people choose to participate, the stakes are intense, and they


motivate critique precisely because the outcome of the project mattersenough to those involved that they want and need to judge their own effortsand assess the work of their peers. Stakes create boundaries, limits, andparameters. In Simon’s case, he knew that he had only one shot at cap-turing the scene he needed in that alley at midnight, given the group’sincredibly tight production schedule and limited resources. It was seriousbusiness, with a whole group of peers counting on his ability to comethrough for this shoot, and with a finished piece alongside which their ownwork would be displayed. Likewise, at Youth Radio, the nonnegotiabledeadlines associated with broadcasts added a sense of urgency to the cri-tique surrounding Rafael’s poem. Very little mattered more to the youngpeople and the adults involved than their own sense of professional andcreative integrity and an ability to deliver the highest quality product toprofessional editors and to the listeners who ultimately hear the stories onthe air. Researchers have used the term safe havens to denote effective artslearning environments (Davis et al., 1993), and that term is apt, in a par-adoxical sense, because it describes spaces that allow young people to ex-periment with the absence of safety inherent in the act of committing to aproject that actually matters to them and to a wider audience.

2. METASTANDARDS ARE SUBJECTED TO REVIEW

Critique takes root in places where criteria for judging the work underreview are themselves subjected to scrutiny. This observation is drawn outby Fleetwood (2005) in her analysis of youth-based media arts organiza-tions. Her analysis suggests that youth-adult collaboratives do not alwaysuse critique to the fullest as a way to examine the tacit ideologies guidingjudgments about the work as it develops (see also Paley, 1995). When theassumptions shaping production escape joint scrutiny, critique can only re-fine existing or accepted practices rather than unsettle those processes andpoint to new and potentially more provocative directions for creative in-tellectual and political work. The capacity to expose and rework meta-standards requires considerable sophistication. In this sense, a very basiccondition for critique is serious focus on skill development—which mightmean ‘‘old-school’’ lecture and demonstration formats alongside hands-onexperimentation—for novices within a given realm of practice to developthe critical literacies they need to interrogate the basic assumptions and‘‘received knowledge’’ driving work in their field.

3. ACCOUNTABILITY IS MUTUAL AND INTERACTIVELY SUSTAINED

Accountability is another buzzword in education, typically framed as a sys-tematic process of evaluating whether an investment has paid off, and then


assigning consequences—for example, determining if reallocations of aschool’s budget yield elevated test scores and making future funding de-cisions on that basis. But in environments that promote critique, account-ability is a moment-to-moment affair. Certainly, young people in critique areaccountable to outside forces, and in fact, this specter of public release iscritical, as we have seen. They are also accountable to one another, oftenquite literally—as is the case with Cutaway, where the young people act inand ‘‘crew’’ for each other’s productions and where the movies need to linkthematically and aesthetically for the installation to work. In art forms suchas dance, a breakdown in peer accountability can have immediate physicalconsequences if the person supporting another person’s body has nottrained sufficiently to carry the weight. Accountability can also play out inthe imaginative and ideological realm—within projects such as Youth Ra-dio’s—in which there is an expectation that the final production or exhi-bition will have a specific impact, whether in the media world, in politicalterms, or both. Young people negotiate the details of accountability withinmoments of display—stopping, seeing, assessing, starting again—duringthe course of production. All this occurs through the immediacy ofmoment-to-moment interactions.

4. INTERDISCIPLINARITY IS MANDATORY

It is commonplace in education to think in terms of mastery within singledisciplines. Standardized test-driven learning environments foster this kindof compartmentalization because paper-and-pencil tests lend themselves to,and help produce, clearly circumscribed bodies of knowledge for meas-urement. But critique-rich environments very often combine multiple dis-ciplines within collaborative projects. The interdisciplinary character ofCutaway is obvious. It involves sound design, visual expression, perfor-mance, sculpture, and architecture. At Youth Radio, central to the critiquesurrounding Rafael’s poem was a process of moving between expressivegenres, specifically spoken-word performance and broadcast journalism,and the effort to find a compelling narrative mode that integrated aspects ofthe two raised new questions that fueled the critique. Using a more ex-pansive definition of disciplines, there are demands that extend acrosssymbol systems and fields of practice. At Cutaway, young people wrotecharacter and scene synopses, drew storyboards, talked through differentoptions, dramatized scenarios, viewed examples of work by establishedartists, and so on. They also carried responsibility for tasks that we do notnormally associate with the arts as ‘‘pure’’ disciplines at all; for example,they researched the legalities of taping minors without parental permission,created strategies for obtaining additional equipment, prepared promo-tional materials, taught new skills to other young people, and so on. At


Youth Radio, young people’s participation in various roles within the or-ganization, including making presentations to funders and calling meetingsto shape and respond to changes in agency policy, exposes them to lead-ership structures, internal communication practices, and organizationallearning that demand the exercise of judgment, and hence participation incritique. Working alone and silently in a studio environment could not befurther from the reality of how learning operated in, and was integratedwithin, these places where youth jointly produce exhibitions and perform-ances for outside audiences.

Taken together, these conditions for critique inform efforts to engageyoung people in assessment of their own work and that of their peers—aprocess that has been linked with improved products and reflections—inways that are embedded within the demands of joint production.

IMPLICATIONS FOR LEARNING THEORY: LEARNINGAS PRODUCTION

These conditions for critique highlight the idea of production as both adriving interactive goal and guiding conceptual metaphor through which totheorize learning. Learning is not, in the episodes examined here, a set ofinformation or skills that young people acquire and simply carry with themfrom one situation to the next. It is not, in other words, a thing to possess.Moreover, learning cannot be reduced to an everyday, natural property ofordinary life—a thing that simply happens in the course of any activity. Butlearning is also more than a momentary or situated cognition accomplishedwithin a specific context. It is more, in other words, than a thing to do(Chaiklin & Lave, 1996; Sfard, 1998). Learning is, in its manifestationthrough critique, a way of making. Young people in critique are constantlyproducing contexts for learning, producing situations in which they need toact and judge. And they are constantly reorganizing their environments inways that extend their situated cognition beyond a given here and now—avideo shoot in an alley within a crisis of choreography, an e-mailed refusal tomake changes to a poem—into a future beyond that moment. In Simon’sfight scene, as an example, moments of critique are steeped in aestheticconcerns—lighting, narrative, conceits of realism, building of suspensethrough sound effects and music, and so on. They are also driven by ne-gotiations of authority, with the Cutaway group tacitly agreeing that Cleocan stick her hand in Chanpory’s face and tell him, essentially, to shut up.Learning as production means participating in countless fleeting momentsof judgment that fuel further action, engagement, and experimentation.

Critique asks learners to remake the situations in which they participateby introducing imagined voices, times, and spaces—moving the interaction


from its literal confines into hypothetical scenarios—a scene that is com-pelling to audiences or one that just ‘‘looks stupid,’’ to use Simon’s example.The language of critique in particular is what linguists might call temporallytranscendent, to the extent that interlocutors ‘‘recontextualize’’ past experi-ence and ‘‘precontextualize’’ future possibilities (Ochs & Jacoby, 1997)—away of feeding forward and not just feeding back. Whereas learning in-habits ‘‘communities of practice,’’ critique draws young people to producesituations that transcend boundaries of voice, time, and space. This processrequires not distancing oneself from present activity by resorting to de-contextualized or transferable bits of information (Chaiklin & Lave, 1996).Rather, the process involves producing imaginary conditions and contin-gencies and projecting them beyond the immediate through moment-to-moment interaction.

IMPLICATIONS FOR RESEARCH ON LEARNING

These theoretical considerations carry implications for the production oflearning, but they also, by way of closing, contain new insights for theproduction of research. Of particular relevance are instances of critiqueinvolving young people working in collaboration with adults to produceoriginal work for distribution to real audiences in the hopes of having a realimpact. Young people and adults working along these lines offer a modelfor scholars who want to work with youth as agents, and not only objects, ofresearch. Cultural studies scholars and education researchers increasinglyseek methods that move beyond the superficial use of youth for access,quotation, consultation, and approval of findings (Fine, 1994; Heath,1998a; Reason & Bradbury, 2001). In an article on the state of youth studiesas an interdisciplinary field, Cohen and Ainley (2000) highlighted newmethodological innovations—audio and video diaries, photo-mapping,story-making—all used in an effort to find a space for meaningful, andrelatively reciprocal, scholar-youth collaborations (see also Lipsitz, 2001;Tuhiwai Smith, 1999).

At places like Cutaway and Youth Radio, young media artists, workingwith adults, observe and ask questions about environments relevant to theirstories. They record their surroundings and interactions using varioustechnologies, and then they develop imaginative ways to tell their stories toan audience. These phases of work give a sense of new ways that scholarsmight involve young people in the making of research as questioners, datagatherers, analysts, and creative producers (Soep, 2003). Arts collaborat-ives, in which youth and adults jointly produce original narratives—video,radio, poetry, and so on—are resources for researchers who initiate edu-cational analysis together with young people.14 As Rafael reminded us,


creating a message, whether it fills the space of a museum gallery, broad-casts over audio airwaves, or materializes as an academic text, means con-veying a significant truth. To do so requires addressing the possibilities ofmanipulations operating on various levels, having the courage and re-sources to take them on, and figuring out a way to say something mean-ingful and multivocal. Innovative educational researchers have anopportunity to bring young people’s epistemologies into conversation withtheir own ideas and those drawn from published literature—using ‘‘youthvoices’’ not as raw materials to be interpreted but as already developedtheories to be reckoned, and sometimes wrestled, with—a process that fuelsfurther analysis. Such an approach requires a willingness on the part ofadults to subject ourselves to critique on the level of the stakes we assert, thestandards we hold dear, to whom and what we are accountable, and in whatbreadth of roles we do and do not allow youth participation. Methodolog-ical projects carried out in the spirit of youth collaboration also have thepotential to socialize young people into professional fields of inquiry, mov-ing well beyond token inclusion of youth as junior field workers or in-formants. This model for research is not possible without an ongoingprocess of critique. And so this analysis of critique ends with a view of howperhaps to begin again, with a revised approach to research that takescritique not only as an object of study but also as a key dimension of theprocess of investigation itself, involving young people and adults workingtogether to produce new understandings of learning.

Earlier versions of this article were delivered in two lectures: one as part of Harvard Uni-versity’s John Landrum Bryant Lecture and Performance Series (2002) and one at the Uni-versity of California at Berkeley’s program in Language and Literacy, Society and Culture(2003). I would like to thank Shirley Brice Heath, Jessica Davis, Leisy Wyman, RubenGaztambide-Fernandez, and Ray McDermott for comments on drafts of this article.

Notes

1 It must be noted, however, that different contexts invite different modes of participa-tion; for example, silence can itself serve as a potent response to a previously uttered assess-ment, and in this sense, not speaking can be its own form of participation.

2 Cutaway is a pseudonym, and I have also replaced the names of all individuals involvedin this organization. In the case of Youth Radio and Youth Speaks, however, with permission, Iuse actual names.

3 Although college or university visual arts classrooms and studios are certainly not aloneas sites for peer critique, as Ruben Gaztambide-Fernandez has pointed out in his comments onthis article, critique should not be taken as an automatic feature of arts learning in general. Oneneed only think of the harrowing experiences that youngsters may have with a domineeringballet or piano instructor. And in traditional theater settings, often the director controls the flowof assessment through the delivery of ‘‘notes,’’ and actors are prohibited (at least officially)


from commenting on one another’s efforts. Experimental theater often unsettles these con-ventions by engaging actors in both the performance and direction of emerging works.

4 Also relevant are studies of a practice similar to arts critique in writing pedagogy—the‘‘workshop,’’ where students review one another’s essays. See, for example, Cazden, 1996;Delpit, 1986; DiPardo & Freedman, 1988; Gutierrez, 1992; Lensmire, 1998; Leverenz, 1994;Reyes, 1992; Sperling, 1996.

5 For the following discussion of three ways to conceive of learning, I am indebted to RayMcDermott (personal communication, 2001).

6 That research investigated youth organizations primarily in working-class neighbor-hoods within urban centers, midsized towns, and rural areas—places ranging from tumblingteams and midnight basketball leagues to agriculture organizations, improv theater groups,and peer tutoring centers (Heath & McLaughlin, 1987, 1993, 1994a, 1994b).

7 Site selection for my in-depth research was based on five criteria: (1) these sites operatedoutside of schools, thereby illuminating learning beyond traditional classrooms; (2) they cen-tered on the arts and thus featured projects that involved the negotiation of personal andcollective meanings and the risk of disclosure; (3) they focused on sustained projects in whichcritique extended through periods of planning, preparation, practice, and performance—cycles associated with effective youth-based learning environments (Heath & McLaughlin,1993); (4) they were tuition free and involved ethnically and economically diverse groups ofyouth and instructors who availed themselves to the generation of learning theory based on thelives of young people who bring a complex range of experiences to the work at hand and donot necessarily conform to habits of speech or development that mark learning environmentscomprising White middle-class learners and teachers; and (5) participants at these sitesexpressed interest in my observing, and in some cases joining, their activities.

8 Cutaway students were Cleo (Chinese American), Simon (Chinese American), Jason(Filipino American), Alison (Caucasian), Jamal (African American), Cassie (Chinese American/Caucasian/Latino), Claire (Korean American), and Lydia (Vietnamese American). Chanporyand Bryant, two friends who acted in one of the movies that Cutaway produced, were bothAsian/Pacific Islander. With the exception of Claire and Alison, all attended urban public orparochial schools.

9 In this transcript, brackets mark the beginning of a passage where two young peoplespeak simultaneously.

10 Along with these responsibilities for judgment come experiences with collaborativeleadership, as documented by Roach and her colleagues (1999).

11 BET is Black Entertainment Television; MTV is Music Television; PYTs are ‘‘PrettyYoung Things.’’

12 This does not, of course, mean that young people’s ‘‘raw’’ voices should be left as is,without the benefit of thoughtful response from peers and adults, only that these kinds ofdecisions need to be made on a case-by-case basis and that even a final outcome with fewchanges to the finished work can sometimes mask a more complex process of negotiation.

13 For this important line of questioning, I am indebted to Elliot Eisner (personal com-munication, 2000).

14 These sorts of projects also have relevance to proposals in the literature regardingartistically grounded approaches to research (see, for example, Eisner, 1995; Lightfoot & Davis,1998).

References

Anderson, J., Reder, L., & Simon, H. (2000). Applications and misapplications of cognitivepsychology to mathematics instruction. Texas Educational Review, 1(2), 29–49.


Barrett, T. (1988). A comparison of the goals of studio professors conducting critiques and arteducation goals for teaching criticism. Studies in Art Education, 30, 22–27.

Barrett, T. (1994). Principles for interpreting art. Art Education, 47(5), 8–13.Barrett, T. (1997). Talking about student art. Worcester, MA: Davis.Barrett, T. (2000). Studio critiques of student art: As they are, as they could be with mentoring.

Theory Into Practice, 39, 29–35.Blythe, T., Allen, D., & Schieffelin Powell, B. (1999). Looking together at student work: A companion

guide to assessing student learning. New York: Teachers College Press.Bulka, J. (1996). You call that art? A critique of the critique. The New Art Examiner, 23(6), 22–25.Campbell, S. (2001). Shouts in the dark: Community arts organizations for students in rural

schools with ‘‘urban’’ problems. Education and Urban Society, 33, 445–456.Cazden, C. (1996). Selective traditions: Readings of Vygotsky in writing pedagogy. In D. Hicks

(Ed.), Discourse, learning, and schooling (pp. 165–188). New York: Cambridge UniversityPress.

Cazden, C. (2001). Classroom discourse: The language of teaching and learning. Portsmouth, NH:Heinemann.

Chaiklin, S., & Lave, J. (1996). Understanding practice: perspectives on activity and context. Cam-bridge, UK: Cambridge University Press.

Cohen, P., & Ainley, P. (2000). In the country of the blind? Youth studies and cultural studies inBritain. Journal of Youth Studies, 3, 79–95.

Cohen, E., Lotan, R., Scarloss, B., Schultz, S., & Abram, P. (2002). Can groups learn? TeachersCollege Record, 104, 1045–1068.

Cotton, C. (1981). Critique: A middle school evaluation system. School Arts, 80(5), 46–47.Darling-Hammond, L., Ancess, J., & Falk, B. (1995). Authentic assessment in action: Studies of

schools and students at work. New York: Teachers College Press.Davis, J. (1998). Everything old is new again: Self-assessment as tradition in community art

centers. In R. E. Stake, K. G. Congdon, & D. Boughton (Eds.), Advances in program eval-uation: Volume 4. Evaluating art education programs in community centers: International perspectiveson problems of conception and practice (pp. 117–132). Stamford, CT: JAI Press.

Davis, J., Soep, E., Maira, S., Remba, N., & Putnoi, D. (1993). Safe havens: Educational effec-tiveness at community art centers that focus on education in economically disadvantaged neighborhoods.Cambridge, MA: Harvard Project Zero.

Delandshere, G. (2002). Assessment as inquiry. Teachers College Record, 104, 1461–1484.Delpit, L. (1986). Skills and other dilemmas of a progressive black educator. Harvard Educa-

tional Review, 56, 379–385.DiPardo, A., & Freedman, S. (1988). Peer response groups in the writing classroom: Theo-

retical foundations and new directions. Review of Educational Research, 58, 119–147.Eisner, E. (1992). The misunderstood role of the arts in human development. Phi Delta Kappan,

73(8), 591–595.Eisner, E. W. (1995). What artistically crafted research can help us to understand about schools.

Educational Theory, 45, 1–7.Ende-Saxe, S. (1990). The elementary critique. School Arts, 90(3), 22–23.Filer, A. (2000). Assessment: Social practice and social product. London: RoutledgeFalmer Press.Fine, M. (1994). Dis-stance and other stances: Negotiations of power inside feminist research.

In A. Gitlin (Ed.), Power and method: Political activism and educational research (pp. 13–35). NewYork: Routledge.

Fleetwood, N. (2005). Authenticating practices: Producing ‘‘the real’’ in youth videos. InS. Maira & E. Soep (Eds.), Youthscapes: The Popular, the National, the Global. Philadelphia:University of Pennsylvania Press, 155–172.

Fox, R. (2001). Constructivism examined. Oxford Review of Education, 27, 23–34.


Goodwin, C., & Goodwin, M. H. (1992). Assessments and the construction of context. InA. Duranti & C. Goodwin (Eds.), Rethinking context: language as an interactive phenomenon(pp. 147–190). New York: Cambridge University Press.

Greeno, J. (1998). The situativity of knowing, learning and research. American Psychologist,53(1), 5–26.

Gutierrez, K. (1992). A comparison of instructional contexts in writing process classrooms withLatino children. Education and Urban Society, 24, 244–262.

Hacking, I. (2000). The social construction of what? Cambridge, MA: Harvard University Press.Heath, S. B. (1993). Inner-city life through drama: Imagining the language classroom. TESOL

Quarterly, 27, 177–192.Heath, S. B. (1998a). The arts in the nonschool hours: Strategic opportunities for meeting the

educational, civic learning, and job training goals of America’s youth. Research reportpresented to the President’s Committee on the Arts and Humanities, March 2. Washington,DC.

Heath, S. B. (1998b). Working through language. In S. Hoyle & C. T. Adger (Eds.), Kids talk:Strategic language use in later childhood (pp. 217–240). New York: Oxford University Press.

Heath, S. B. (2001). Three’s not a crowd: plans, roles, and focus in the arts. Educational Re-searcher, 30(7), 10–17.

Heath, S. B., & Ball, A. (1993). Dances of identity: Finding an ethnic self in the arts. In S. B.Heath & M. McLaughlin (Eds.), Identity and inner city youth: Beyond ethnicity and gender (pp.69–93). New York: Teachers College Press.

Heath, S. B., & McLaughlin, M. (1987). A child resource policy: Moving beyond dependenceon school and family. Phi Delta Kappan, 68, 576–580.

Heath, S. B., & McLaughlin, M. (1993). Identity and inner city youth: Beyond ethnicity and gender.New York: Teachers College Press.

Heath, S. B., & McLaughlin, M. (1994a). Learning for anything everyday. Journal of CurriculumStudies, 26, 549–567.

Heath, S. B., & McLaughlin, M. (1994b). The best of both worlds: Connecting schools andcommunity youth organizations for all-day, all-year learning. Educational AdministrationQuarterly, 30, 278–300.

Heath, S. B., & Soep, E. (1998). Youth development and the arts in the non-school hours.Grantmakers for the Arts, 9(1), 9–32.

Hogan, K. (2002). Pitfalls of community-based learning: How power dynamics limit adoles-cents’ trajectories of growth and participation. Teachers College Record, 104, 586–624.

House, N. (2001). Critiques in the K-12 classroom. NAEA Advisory. http://www.nea-reston.org/pdf/2crit.pdf. Retrieved January 27, 2006.

Hull, G., & Schultz, K. (2001). Literacy and learning out of school: A review of theory andresearch. Review of Educational Research, 71, 575–611.

Hull, G., & Schultz, K. (2002). School’s out: Bridging out-of-school literacies with classroom practice.New York: Teachers College Press.

James, P. (1996). The construction of learning and teaching in a sculpture studio class. Studies inArts Education, 37(3), 145–159.

Lave, J., & Wenger, E. (1991). Situated learning: legitimate peripheral participation. Cambridge, UK:Cambridge University Press.

Lee, C. (2001). Signifying in the zone of proximal development. In C. Lee & P. Smagorinsky(Eds.), Vygotskian perspectives on literacy research: Constructing meaning through collaborative in-quiry (pp. 191–225). Cambridge, UK: Cambridge University Press.

Lensmire, T. (1998). Rewriting student voice. Journal of Curriculum Studies, 30, 251–291.Leverenz, C. S. (1994). Peer response in the multicultural composition classroom: Disensus—a

dream (deferred). Journal of Advanced Composition, 14, 167–186.Lightfoot, S. L., & Davis, J. (1998). The art and science of portraiture. San Francisco: Jossey-Bass.


Linde, C. (1997). Evaluation as linguistic structure and social practice. In B. L. Gunnarsson,P. Linnell, & B. Nordberg (Eds.), The construction of professional discourse (pp. 151–172).London: Longman.

Lipsitz, G. (2001). American studies in a moment of danger. Minneapolis: University of MinnesotaPress.

Mahiri, J. (2003). What they don’t learn in school: Literacy in the lives of urban youth. New York: PeterLang.

Noam, G., Biancarosa, G., & Dechausay, N. (2002). Afterschool education: Approaches to an emerg-ing field. Cambridge, MA: Harvard Education Press.

Ochs, E., & Jacoby, S. (1997). Down to the wire: The cultural clock of physicists and thediscourse of consensus. Language in Society, 26, 479–505.

Paley, N. (1995). Finding art’s place: Experiments in contemporary education and culture. New York:Routledge.

Papert, S. (1991). Situating constructionism. In S. Papert & I. Harel (Eds.), Constructionism (pp.1–12). Norwood, NJ: Ablex Publishing Company.

Reason, P., & Bradbury, H. (2001). Handbook of action research. Thousand Oaks, CA: Sage.Reyes, M. (1992). Challenge venerable assumptions: Literacy instruction for linguistically dif-

ferent students. Harvard Educational Review, 62, 427–446.Roach, A., Wyman, L., Brookes, H., Chavez, C., Heath, S. B., & Valdes, G. 1999. Leadership

giftedness: Models revisited. Gifted Child Quarterly, 43(1), 13–24.Rogers, J. (1996). The concept of framing and its role in teacher-student negotiations during desk

critiques in the architectural design studio (Unpublished doctoral dissertation, Texas A & MUniversity, College Station, TX.

Rogoff, B. (1994). Developing understanding of the idea of communities of learners. In Mind,Culture, and Activity, 1(4), 209–229.

Roth, R. (1999). The crit. Art Journal, 58, 32–35.Sfard, A. (1998). On two metaphors for learning and the dangers of choosing just one. Ed-

ucational Researcher, 27(2), 4–13.Siegesmund, R. (2002). Bringing accountability to elementary art. Kappa Delta Pi Record, 39,

24–28.Soep, E. (1996). An art in itself: Youth development through critique. New Designs for Youth

Development, 12(4), 42–46.Soep, E. (2000). To make things with words: Critique and the production of learning (Unpublished

doctoral dissertation, Stanford University, Stanford, CA.Soep, E. (2002). Abnormal educational vision: Art in the city beyond school. Kappa Delta Pi

Record, 39, 12–16.Soep, E. (2003). Learning about research from youth media artists. Penn GSE Perspectives on

Urban Education, 2(1). http://www.urbanedjournal.org/archive/issue3/commentaries/comment0004.html. Retrieved January 27, 2006.

Soep, E. (2004). Visualizing judgment: Self and peer assessment in visual arts education. InE. Eisner & M. Day (Eds.), Handbook of research and policy in the visual arts.

Soep, E. (2005). Making hard-core masculinity: Teenage boys playing house. InS. Maira & E. Soep (Eds.), Youthscapes: The popular, the national, the global (pp. 667–687).Mahwah, NJ: Lawrence Erlbaum Publishers.

Sperling, M. (1996). Revisiting the writing-speaking connection: Challenges for research onwriting and writing instruction. Review of Educational Research, 66, 53–86.

Tannock, S. (1998). Noisy talk: Conversation and collaboration in a youth writing group. InS. Hoyle & C. T. Adger (Eds.), Kids talk: Strategic language use in later childhood (pp. 241–266).New York: Oxford University Press.

Thibault, M. (1997). Personal Communication.


Trend, D. (1997). The fine art of teaching. In H. Giroux & P. Shannon (Eds.), Education andcultural studies: Toward a performative practice (pp. 249–258). New York: Routledge.

Tuhiwai Smith, L. (1999). Decolonizing methodologies: Research and indigenous peoples. London: ZedBooks.

Varenne, H., & McDermott, R. (1999). Successful failure: The school America builds. Boulder, CO:Westview Press.

Weiss, L., & Fine, M. (2000). Construction sites: Excavating race, class, and gender among urban youth.New York: Teachers College Press.

Wernik, U. (1985). Psychological aspects of criticism in an academy of art and design. Journal ofCreative Behavior, 19, 194–201.

Young, M. F. D. (2000). Rescuing the sociology of educational knowledge from the extremes ofvoice discourse: Towards a new theoretical basis for the sociology of curriculum. BritishJournal of Sociology of Education, 21, 523–535.

ELISABETH SOEP received her PhD from Stanford University and hastaught at University of California, Berkeley, and San Francisco StateUniversity. Her research, featured in national and international journals,centers on youth learning, language, and cultural production in non-schoolsettings. Dr. Soep recently coedited Youthscapes: The Popular, the National, theGlobal (with S. Maira, 2005, University of Pennsylvania Press). She iscurrently working on a new book, Making the News: Youth Radio, Education,and Media Culture (with V. Chavez, University of California Press). Soep isthe education director and a senior producer at Youth Radio, where shecollaborates with young media artists on stories for local and nationaloutlets, including National Public Radio. Youth Radio has been recentlyrecognized with honors, including the George Foster Peabody and EdwardR. Murrow Awards.


Critique: Assessment and the Production of Learning

Documents

Transcript of Critique: Assessment and the Production of Learning