Assignment 2—Annotated Outline A Comprehensive...

25
Assignment 2—Annotated Outline A Comprehensive Approach to Understanding and Implementing Student Self-Assessment by Ewing Coleman Green EDD 9200 CRN 35456 Trends and Issues: Society, the Individual and the Professions Nova Southeastern University March 23, 2012

Transcript of Assignment 2—Annotated Outline A Comprehensive...

Assignment 2—Annotated Outline A Comprehensive Approach to Understanding and Implementing Student Self-Assessment

by Ewing Coleman Green EDD 9200 CRN 35456

Trends and Issues: Society, the Individual and the Professions

Nova Southeastern University March 23, 2012

2

Since Black & Wiliam’s 1998 seminal work on formative assessment, educational institutions from kindergarten to college have gradually embraced the notion that assessment can be more than a singular teacher-driven event to determine an end-of-unit or course grade. Formative assessment is becoming a standard practice along the learning journey, during formation of understanding, to inform both teacher and student as to areas of strength and areas in need of reinforcement. Self-assessment is a relatively new phenomenon and an outgrowth of formative assessment where students take more direct, active responsibility for their learning. McMillan’s 2008 and Andrade’s 2009 and 2010 work made the argument that student self-assessment (SSA) practices have multiple benefits including self-efficacy, metacognition, fostering intrinsic motivation and habituating lifelong learning. The literature is becoming richer as researchers investigate these significant and complex areas of education and human development. To date, however, given the relative infancy of the self-assessment field, studies have focused more on making the case for SSA, defining its characteristics, and investigating relatively narrow aspects of design and implementation. Comprehensive meta-analyses have yet to be conducted across the SSA landscape. Therefore, holistic investigations now need to be conducted that are founded in the latest SSA research, rationale, approaches to implementation, and effective tools and student training, that will serve to advance the state of the art in SSA and add validity to the significant learning and human development benefit claims made by leading researchers. Topic: A Comprehensive Approach to Understanding and Implementing Student Self- Assessment

I. Introduction a. Importance of lifelong learning

Kirby, Knapper, Lamon & Egnatoff (2010) stated that lifelong learning is widely espoused as a goal for education and an essential workplace component. The term is used extensively in education, government and society. In addition, it should be viewed as life-wide, beyond formal education and include social, recreational, and informal education. The authors cite Candy, Crebert and O’Leary (1994) and Knapper and Cropley (2000) who suggest that effective lifelong learners set goals, apply appropriate knowledge and skills, engage in self-direction and self-evaluation, locate required information, and adapt their learning strategies to different conditions. Heritage (2009) reported that knowledge and information are growing exponentially on a global basis. Thus, Learning how to Learn (LtL) is a critical life and career skill that students must develop before leaving formal education. New developments in the science of learning are finding benefits from greater student control of their own learning. Metacognitive approaches that increase transference of learning to new settings include self-assessment, sense-making, and reflection.  

3

Donham (2010) argued that lifelong learning is a personal, self-directed process that begins with self-directed questions, includes self-directed inquiry and explorations, and concludes with self-measured success.  

b. Importance of self-efficacy Cole & Denzine (2004) suggested that self-efficacy provides the strongest evidence of student motivation. Students’ perceptions of their abilities and confidence for completing a task are strongly related to their motivation within a given situation. Academic self-efficacy repeatedly surfaces as a critical variable in understanding students’ academic experiences and performance. The authors argue that tutor training should address principles of self-efficacy theory as well as the identification of explanatory styles. Hughes (2010) stated that self-efficacy is the belief that a person has regarding their ability to successfully perform a certain task or manage a given situation. A person’s level of self-efficacy influences the choice of activities a person will engage in, how much effort they exert, the extent of persistence upon encountering obstacles, and the level of performance they aspire to. Teachers can build student self-efficacy through formative assessment by focusing more on the process of learning as opposed to a singular outcome grade. This is particularly beneficial to struggling learners. Specifically, teachers can help students set realistic goals and provide ongoing support throughout instruction, assessment, and feedback. Brady-Amoon & Fuentes (2011) reported that building on self-beliefs can promote healthy adjustment and performance across a wide range of settings including school and work. According to social-cognitive theory, people who have the ability to act and believe that their actions will produce desired outcomes are more motivated to act and act in ways that are more likely to produce desired outcomes than people who do not believe their efforts will be successful. Margolis & McCabe (2004) suggested that struggling learners tend to avoid or give up on tasks where they have previously been unsuccessful. Teachers are in a strong position to favorably develop student self-efficacy by linking new work to recent successes, teaching needed learning strategies, reinforcing effort and persistence, and helping students identify and create personally important goals. Classwork needs to be at a proper instructional level and homework at an appropriate level of independence. Students need to perceive academics as challenging, not frustrating.

c. Motivation

i. Student learning perspective Zimmerman (2002), an early contributor to the field of self-regulation, defined self-regulation as a self-directive process by

4

which learners transform their mental abilities into academic skills,  not a mental ability or academic skill in itself. He argues that learning is a student-driven, proactive process versus a passive  event as a reaction to teaching, and that self-regulated learners generate thoughts, feelings, and behaviors that are oriented toward the achievement of goals. Therefore, self-regulated learners are self-reflective, motivated, and adaptive, which results in their being more likely to succeed academically as well as view their futures optimistically, benefiting their development as lifelong learners. The author shares insights from the field of social learning psychology, noting three cyclical phases that underlie self-regulatory processes: forethought, performance, and self-reflection. The author noted at the time that research suggested few teachers effectively prepare students to learn on their own, such as by developing self-evaluation or reflection. Kizilgunes, Tekkaya & Sungur (2009) found that students who view knowledge as continually under development and handed down by authority were more likely to have higher levels of self-efficacy and have stronger orientation toward learning and performance goals. Research has revealed two primary learning approaches: meaning (resulting in deep understanding) and rote (surface level learning). Deep or meaningful learning occurs when students relate new learnings to prior understanding and build upon them. Self-efficacy and mastery-goal orientation are positively related to deep approaches to learning and to more persistence and effort. Students who believe that ability is malleable and can be improved through effort tend to have more adaptive motivational beliefs. They tend to attribute success or failure to their effort, something they can control. Learners employing the meaningful learning approach are better equipped to maintain knowledge and apply understanding to new situations. Educators can help students significantly by providing opportunities for them to control some of their learning. In addition, individual student improvement, learning, and understanding needs to be emphasized and students should be able to realize the link between their effort and accomplishments. Schweinle, Meyer & Turner (2006) reported that students with increased levels of learning autonomy tend to be more intrinsically motivated. Teachers should help students develop multiple strategies in solving problems and help students take more personal control over their learning.

5

ii. Teacher Ali (2009) found that teacher expectations correlate with student self-perception, motivation, and achievement. Teachers can motivate their students to learn by boosting their self-esteem through opportunities that build mutual trust and instill a belief in  the student’s ability to succeed. These inward beliefs are central to motivation and are vital for lifelong success. The author suggests that it is important for teachers to provide the proper challenge level based on student needs. Den Brok, Levy & Brekelmans (2005) argued that teacher interpersonal behavior effects student motivation. Their work focused on proximity (teacher cooperation versus opposition) and influence (teacher dominance versus submissiveness). Results show that proximity has a greater impact on student motivation than influence. Specifically, teachers should maximize helpful, friendly, and understanding behaviors and minimize dissatisfied and correcting behaviors. Schweinle, Meyer & Turner (2006) used flow theory to study the relationship between student motivation and teacher instructional practice. Results indicated that student motivation is increased when teachers provide substantive feedback, clarify concepts, encourage autonomy, cooperation, and social relatedness, and emphasize learning for its own sake. Also, motivation is enhanced by emphasizing a balance of challenge and skill, supporting self-efficacy and value for mathematics, and fostering positive affect. Flow theory seeks to explain the experiences of intrinsically motivated people. The authors argue that optimal experience, or flow, occurs when someone perceives the challenges in a certain situation and his or her skills as balanced and above average. When challenges and skills are unbalanced, such as when challenges outpace skills, an activity is not necessarily rewarding and could evoke anxiety. The authors also found that when teacher feedback was frequent, elaborative, positive, and used to help develop student understanding, students report higher affect, efficacy and importance. Furthermore, emphasizing learning and understanding rather than memorizing, treating mistakes as learning opportunities, supporting autonomy, and stressing the importance of mathematics can encourage more positive affect, efficacy and value.

iii. Anxiety

Kesici & Erdogan (2010) reported that studies have shown anxiety can be a significant barrier to student achievement in mathematics, and most directly related to students’ math ability perceptions,

6

performance expectancies, and value perceptions. Students’ achievement motivation appears to play a significant role in the existence of mathematics anxiety. The authors discuss Achievement Motivation Theory, which focuses on the required tendency of the individual achieving success on a given task and avoid underachievement. Results showed that higher mathematics achievement motivation was accompanied by higher anxiety than students possessing low achievement motivation. Social comparison also impacts mathematics anxiety as students with negative self-esteem experience more mathematics anxiety compared to students with more positive self-esteem. Students can maximize mathematics success by minimizing social comparison and focus on developing positive self-concept and increased self-efficacy. Shores & Shannon (2007) suggested that motivation, anxiety and academic performance are interrelated. Levels of self-efficacy and worry positively correlate with academic performance. Higher levels of anxiety such as during assessments, however, lead to lower performance. These factors interrelate to influence how learners self-regulate in an academic domain.

iv. Time perspective

Kauffman and Husman (2004) argued that there is a relationship between temporal perspective, motivation, and self-regulation. They also believe there are gaps in U.S. and international perspectives on learning and motivation. Flow and self-determination theories are cited as focusing students on being motivated in the present and on the process as opposed to the product. The author offers six articles, including one on FTP (Future Time Perspective), that suggest students’ conceptions of the future have a real and significant influence on their beliefs and motivation, positively influencing their achievement.

v. Family, student characteristics, and interpersonal relationships Anderman and Kaplan (2008) suggested that the umbrella of social motivation includes social relationship variables such as social acceptance, perceived belonging, identification with the school, and culture. There are multifaceted ways in which interpersonal relations and academic motivation impact each other. Halawah (2006) studied the effect of motivation, family environment, and student characteristics on academic achievement. Intelligence and high levels of student motivation and engagement have been shown to positively influence academic success. Some of the family environmental factors included parental willingness

7

to help, confidence in their child’s ability, and expectations of homework completion and future college attendance. Some of the student characteristics they studied included the ability to plan, organize, complete assignments on time, and be self-motivated to complete their work. They noted remarkably high correlation  between motivation and student characteristics and even higher correlation between family environment and student characteristics.

II. Formative assessment

Black & Wiliam (1998), early contributors to the field of formative assessment (FA), defined FA as all activities undertaken by teachers and/or their students which provide information to be used as feedback in modifying the teaching and learning activities in which they are engaged. As such, FA has two distinct actions: the perception by the learner of any gaps between a desired goal and their current state of knowledge, understanding, and/or skill, and the steps taken by the learner to close those gaps in order to attain the desired goal. Self- or peer-assessment may be beneficial to this process. This foundational work in framing and delineating the benefits of FA focused on eight studies from 1986-1997, representing a relatively wide international, grade-level, learner ability, and subject-content area study at the time. Areas of focus included self-efficacy, mastery learning, achievement, intrinsic motivation, and types of evaluation. The authors note that the term FA itself was relatively new at the time in assessment literature so they set about developing a theory of FA and suggested various research prospects and needs. Cauley and McMillan (2010) argued that FA can have a powerful impact on student motivation and achievement. FA includes gathering evidence of student knowledge and understanding, providing feedback to students, and the altering the way that students subsequently learn. Low-level formative assessment is rudimentary and either excludes some characteristics or just introduces each characteristic as an explicit component to be fully developed. High-level formative assessment is marked by a complete dedication to fully integrating the characteristics into teacher and student practice. The authors argue that FA benefits student learning by providing specific ongoing feedback to students while learning is forming and therefore offers the opportunity for both teacher and student to make concrete pedagogical adjustments. They also point out that FA practices are consistent with the constructivist learning theories of learning and motivation. When students focus on improvement and progress, they are more likely to adopt  mastery goals and develop high self-

8

efficacy and expectations for success. Also, when students and teachers  attribute student successes to effort, this attribution supports future successes.

Wang, Wang, Wang & Huang (2006) investigated the effects of formative assessment and learning style on student achievement in a Web-based learning environment. Online assessments, they believe, have several advantages over traditional classroom assessment since they can be taken at any time, taken repeatedly, and provide immediate feedback to help remedy weaknesses in their understanding. Formative assessments can also serve to lower student anxiety if taken prior to summative assessments by allowing students to improve learning before taking assessments that will have a greater impact on their grades. The authors used Kolb’s learning style inventory. They found that both learning style and formative assessment strategy significantly impact student achievement in Web-based learning, suggesting that both should be taken into account in Web-based learning environment design. Generally speaking, the more diverse the Web-based formative assessment strategies, the greater the positive learning effect obtained by the students.

McMillan (2010) stated that years of research has shown that effective instruction depends on many factors, including the style of teaching, objectives, characteristics of students, and the context for learning. FA is most effective in classrooms that emphasize mastery-goal orientation where students are encouraged to seek help, work hard, and have a willingness to accept and use feedback to promote learning. There should be strong emphasis, then, on what teachers do with assessment data and how instructional variables may influence the effectiveness of applying different variations of the entire process of FA. FA strategies can vary widely in format such as after an instructional unit or even minute-by-minute as instruction progresses. The cognitive level of subsequent assessment informs the type of FA from simple knowledge results, focusing on what is remembered, to deep understanding, which involves thinking about one’s knowledge and using knowledge for problem-solving, critical thinking, and reflection. Research shows that low achieving students tend to benefit from immediate specific feedback while for high-achieving students effective feedback is contingent on the level of the task. Grade-level is also a variable in FA design as older students are generally better able to make decisions about what further instruction is needed to help them, and are more adept at using tools such as rubrics to reflect on their  performance. FA involves instructional correctives showing students the discrepancies

9

between their current performance and learning goals with an emphasis on how to close the gap. Two  important skills are supported when using FA for deep understanding: metacognition and self-reflection. Self-regulating students make decisions about what and how they will learn. They actively devise learning strategies that will improve performance. Self-regulation is a broader construct that involves metacognition, self-evaluation, self-reaction, and self-assessment. An emphasis on self-regulation helps students realize that they are responsible for their learning. FA, then, focuses on the process of learning more than the product or outcome. It encourages students to develop self-appraisal and self-management skills that enhance self-directed learning. The learning environment must be student-centered, where students value and actively engage in applying evaluative criteria to review and improve their work. Teachers and students become partners in learning.

III. Student self-assessment

i. Benefits Andrade (2010) suggested that student self-assessment (SSA) takes FA a step further and places the primary focus on students as the definitive source of feedback, not teachers. Student self-regulation and SSA approaches need to be scaffolded. The author argues that self-regulation focuses on how students manage learning processes whereas SSA focuses on student judgments of the products of their learning, with the central purpose of both being feedback students can use to deepen their understanding and improve performance. The author further argues SSA is task-specific, involving students judging the degree to which their work reflects explicitly stated goals or criteria followed by appropriate revisions, while self-evaluation involves students in grading their work, and self-reflection promotes student self-discovery and awareness. Thus, SSA involves three steps: expectation articulation, critique in terms of expectations, and revision. The author states that SSA research to date in mathematics and writing shows promising results. SSA has been shown to help students who do not actively seek help or engage in learning for fear of perceived threats to self-esteem or social embarrassment. Using Zimmerman (2000) and Hattie & Temperley (2007) the author offers a model to synthesize self-regulation and SSA: forethought (Where Am I Going?), performance and control (How am I doing?), and reflection (What’s next?). Feedback that aims at improving students’ strategies and processes as well making improvements at the task at hand are most powerful. Classroom culture is important and needs to be viewed as safe for critique by students in making learning improvements. Students must perceive SSA as valued and

10

valuable, and teachers’ messages about effort, understanding, and grades must be influential.  McMillan and Hearn (2008) argued that when correctly implemented, SSA can promote intrinsic motivation, internally controlled effort, mastery goal orientation, and more meaningful learning. Its powerful impact on student performance, in both classroom assessments and large-scale accountability assessments, empowers students to guide their own learning and internalize the criteria for judging success. SSA occurs when students judge their own work to improve performance as they identify and act on discrepancies between current and desired performance. SSA is supported by these theories: Cognitive and Constructivist Learning and Motivation Theories, Metacognition Theory, and Self-Efficacy Theory. When students set goals that aid their improved understanding, and then identify criteria, self-evaluate their progress toward learning, reflect on their learning, and generate strategies for more learning, they will show improved performance with meaningful motivation.

Cauley and McMillan (2010) reported that SSA is a process in which students monitor and evaluate the nature of their thinking to identify strategies that improve understanding. Self-assessment is a three-step process in which students judge their own work (self-monitor), identify discrepancies between current and desired performance (self-evaluation), and identify and implement further learning activities to enhance their understanding or skills. McMillan (2010) suggested that self-assessment is excellent for older students to form understanding by enabling students to apply targeted feedback on areas that will improve their learning. This serves to enhance student motivation by reinforcing internal attributions and promoting self-efficacy. Students are more able to see that understanding is under their control and will have positive expectations about further learning. Brookhart, Andolina & Zuza (2004) conducted an action research study to determine whether student self-assessment in the service of memorizing math facts tables, a rote activity, would add desirable outcomes besides simple knowledge of math facts. It was expected that the higher-order thinking and metacognitive processes required for self-assessment would turn this rote memory task into an exercise in learning how to learn and contribute to students' mathematical literacy  development. The authors state that the two main benefits of SSA are motivational and cognitive. Motivation theorists suggest that SSA will contribute to feelings of

11

control over one's  own learning, of  choice and agency, and of self-worth. Theorists who focus on the formative feedback loop emphasize that the learning task requires students to compare their performance with desired performance and take steps to close that gap. Accurate appraisal of their own work is a necessary part of this cycle. Results showed that student involvement in their own assessment can benefit student reflection and metacognition skills. As with all learning strategies, student self-assessment needs to be taught, coached, and supported. Donham (2010) suggested that self-assessment is a habit of mind that engages one in metacognition and reflection. Such reflective behavior calls for intentionally reviewing one’s own performance based on criteria that have been internalized and can be thought of as metacognition with an evaluative bent. When learners are metacognitive, they are aware of what they know and what they do not know. They are able to plan a strategy for producing what information is needed, are aware of their own strategies as they problem-solve, and can reflect and evaluate their own productiveness. These behaviors constitute the mindfulness necessary to be self-reliant learners for life. Self-assessment can be a way to give students a sense of ownership of their learning and responsibility for it. Self-assessment is inherently a disposition toward excellence and cultivates the strength and humility to continue to learn. Therefore, the ability to self-assess can serve as a mark of the life-long learner. To believe in self-assessment as essential to lifelong learning is to believe students can and should reflect upon and assess their own performance. Self-assessment involves looking in three directions: back at completed work, down at present work to determine next steps, and forward to the future to apply what has been learned to the next learning opportunity. The author feels that by developing habits of self-assessment students are encouraged to evolve from dependence on others by cultivating the habit of continuous personal learning—an essential life habit in a world of change. Bingham, Holbrook & Meyers (2010) found that even for elementary school learners, self-assessment can return voice and ownership of learning to students. Students become able to engage in self-assessment as their metacognitive abilities (their awareness of their thought processes, strategies, and skills) develop. Children begin to develop metacognitive awareness at an early age as parents, caregivers, and early childhood teachers provide them with feedback regarding their behavior, intelligence, and abilities. This feedback begins to shape a child's sense of self, both as an individual and as a learner, and is the foundation for developing

12

the ongoing processes of self-appraisal.  Research suggests that while general cognitive ability relates strongly to academic achievement, motivation and self-appraisal also contribute to children’s engagement in learning and academic success. The authors include Swartz & Perkins’ (1989) ladder of metacognition that includes these levels of thought: tacit, aware, strategic, and reflective. Reflective self-assessment can contribute to students’ positive self-perception, motivation, and achievement. Kitsantas, Reiser & Doster (2004) studied how two types of goal setting (process vs. outcome), self-evaluation (presence vs. absence) and organizational signals (presence vs. absence) impacted student ability to perform a set of procedural skills. They also examined how these variables influenced a variety of affective outcomes, including students' self-efficacy, satisfaction with their performance, evaluation of the instruction, and attributions of success or lack of success in acquiring the skills. Results indicated that students in the process goal condition reported a higher degree of self-efficacy, more performance satisfaction, and more strategic attributions than students in the outcome goal condition. Moreover, among students who were not directed to self-evaluate their own work, those students in the process goal condition demonstrated a higher level of animation skills, self-efficacy, and satisfaction, and rated the instruction more positively than did their counterparts in the outcome goal group. Self-evaluation also had a positive effect on student skill acquisition, particularly for students in the outcome goal condition. The authors report that this study supports other research in providing evidence that student performance and attitudes can be enhanced by having them focus on process goals as they are learning a procedural skill, especially if the students are not directed to evaluate their own work. The study also indicates that self-evaluation is a valuable learning tool, particularly for students with an outcome goal orientation. Teachers should use strategies that encourage students who are learning a new skill to focus their attention on correctly performing each of the steps required for that skill and to make available to students frequent opportunities for self-evaluation. This may enhance students' self-regulated learning efforts and, consequently, performance and attitudes.    Harada (2010) suggested that the ability to regulate one’s own learning means that a person is intrinsically motivated, strategic in approaching problems, and metacognitive in examining one’s progress. They are also self-modifying as they reflect on experiences and construct new meanings. Self-assessment makes  students more aware of their own journey through the stages of

13

genuine inquiry. Being able to assess one’s own progress contributes to a student’s sense of confidence and self-satisfaction—major elements in motivation. In spite of the benefits associated with self-regulated learning, researchers have demonstrated that a majority of students are not adequately self-regulating. The author suggests that this gap in learning is not surprising since students have few opportunities in school to regulate and assess their learning. Metacognition and reflection are practices that must be thoughtfully and intentionally cultivated in students. Young learners, even in the primary grades, are capable of assessing their work. Heritage (2009) suggested that self-assessment, as a means to improve learning, includes monitoring and managing. Students engaged in self-assessment develop greater responsibility for their learning when they have clear goals and an opportunity within the learning environment to develop and exercise monitoring and managing skills. Students are unlikely to thoughtfully self-assess or self-regulate unless these they know acts will lead to better grades, deeper understanding, and more well-developed skillsets. Therefore, revision and revisiting are essential. Students become more adept and further value SSA with experience and expertise.

Ramirez (2010) argued that self-assessment is an essential component of formative assessment and must be distinguished from the assessment of learning. Formative assessment is assessment for learning and is intended to create independent, reflective learners that can assess their own progress and take actions to fill the gaps between their actual knowledge or understanding and the desired goal. In the author’s study students received three multiple-choice question tests throughout the semester. The course employed a webpage (Moodle platform) with supporting material (PowerPoint slides used in lectures, cases, and problems) accessible to all students. After completing tests students were asked to voluntarily answer a survey intended to rank the factors that could have contributed to their increased success. Among the nine main factors mentioned by the students, the availability of a test for self-assessment before the final test was by far the most frequently mentioned (82.4% of the students). Other factors, like a comfortable place (quiet), more study, and availability of supporting material were considered relevant by 33–38% of the students, while stress was considered a minor issue.

 Carr (2008) reported that many researchers have argued for an emphasis on assessment for learning (formative assessment), rather than assessment of learning (summative assessment). Self-

14

evaluation is a powerful technique for supporting students' involvement in their own learning while also helping teachers gain important information for instructional decision making. When students use self-evaluation, they review and evaluate their own performance by comparing it against some standard. In the process, students must step back and reflect on what and how they learn. Teachers should explicitly teach self-evaluation as they would teach any subject. In teaching students how to self-evaluate the authors suggest that teachers: provide a rationale and explain why self-evaluation is a useful skill to acquire, describe how to self-evaluate by using the format selected, model the technique by thinking aloud while they evaluate their own work, provide opportunities for both guided practice and independent practice, monitor and provide feedback, and teach for generalization and transfer. When both teachers and students become involved in setting goals and evaluating student performance, they become partners in the learning process. This partnership should ultimately have a positive effect on student learning.

ii. Student competence and training Ozogul & Sullivan (2009) studied the effects of teacher-, self-, and peer-evaluation on preservice teachers in writing lesson plans and placed additional emphasis on training from their earlier work. The overall results suggest that when proper student training is provided, self-evaluation and peer-evaluation are appropriate alternatives or complementary approaches to teacher-based formative evaluation of student work. This is especially true when teachers are reluctant or unwilling to spend the amount of time required for both formative evaluation and the final evaluation, typically including grading, of student products. Formative self-evaluation also gives students practice in monitoring and evaluating their own work, a type of self-regulation behavior that over time can become a valuable life skill. The authors attribute the stronger performance of the student-evaluation groups to their training on the evaluation task. Clauss and Geeley (2010) explored of the relationship between student self-assessment skills and different cognitive levels using Bloom’s taxonomy of educational objectives. They predicted that students would have more difficulty accurately self-assessing at higher Bloom levels where students confront more complex, less clearly-structured tasks in synthesis and evaluation. Using knowledge surveys the authors sought to measure self-assessment ability, exam scores, and gains in self-efficacy. Results showed students were better self-assessors at low (knowledge) and high (analysis, synthesis and evaluation) Bloom levels than at

15

intermediate levels (comprehension and application). The authors reported mixed findings regarding knowledge surveys as an effective stand-alone self-assessment tool and suggested that further research is warranted. They do feel, however, that the surveys effectively captured gains in student self-efficacy since by design the surveys prompt metacognition across a wide range of Bloom levels, much more so than is typically present on any single assignment, project, or exam.

Kostons, van Gog & Paas (2010) reported that learner-controlled instruction (LCI) is often found to be less effective for learning than fixed or adaptive system-controlled instruction. One possible reason for that finding is that students, especially novices, might not able to accurately assess their own performance and select tasks that fit their learning needs. Their explorative study investigated the differences in self-assessment and task-selection processes between effective and ineffective learners, measured in terms of learning gains, when studying in an LCI environment. Results indicated that although effective learners could more accurately assess their own performance than ineffective learners, they used the same task aspects to select learning tasks. For effective learners, who were also more accurate self-assessors, the self-assessment criteria predicted subsequent task selection. The authors recommend further research into five training aspects: 1) Whether learners would become more accurate self-assessors if they were trained in the use of relevant criteria and objective standards for performance. Their results suggest that such an increase in self-assessment accuracy might lead to more effective learning in LCI. 2) How learners use certain assessment criteria during task selection. Making learners aware of what assessment criteria are used by effective learners or by effective adaptive instructional systems, and how these are used, might enhance the quality of task selection and foster learning. 3) Whether training in self-assessment and task selection would positively impact the ability to monitor performance while working on the learning tasks, as cognitive skills are built that may guide attention to appropriate task aspects already being performed. The authors argue this may or may not lower the overall cognitive load experienced, but it may direct cognitive load away from ineffective processes for performing the primary and secondary task (i.e., decrease extraneous load) to processes that contribute to performing those tasks. 4) When self-assessment and task-selection skills can be enhanced by training, it should be investigated  whether this actually positively affects learning in LCI. 5) The timing of training— such as whether training should occur before learners engage in LCI using examples that demonstrate how good

16

self-assessors evaluate their performance and task select. Training could also be provided while they are engaging in LCI. The authors cite research on self-regulated learning in hypermedia environments focused on influencing students’ self-regulatory behavior during study, such as by using prompts (Azevedo & Cromley, 2004 and Bannert, 2004) or feedback (Narciss, Proske, & Koerndle, 2007). These procedures not only seemed to positively impact self-regulation processes, but also the primary process of learning. Such guidance for task selection and self-assessment while engaging in LCI should, however, probably be reduced as expertise increases. The authors state that research on instructional guidance shows that more experienced learners no longer benefit from elaborate guidance and might even be hindered by it. Lew, Alwis & Schmidt (2010) conducted a dual study on the accuracy of student ability to self-assess over time and investigated whether self-assessment is more accurate if students believe it contributes to their learning. Results in the first study showed students judged as being more academically competent, measured by GPA, were able to self-assess with greater accuracy as compared to their less competent peers. Over time, students consistently underestimated their own performance in comparison to how they were assessed by peers while overestimating their own performance as compared to grades given by their tutors. The authors cited several possibilities for these discrepancies including that the students may be generally poor assessors (insufficient access to their own learning processes), the students might be good self-assessors but lacked self-assessment experience, the students may be good self-assessors but correlations were weak to moderate because of insufficient data, and the students may not have taken self-assessment seriously. Results in the second study showed no inter-relationship between students’ beliefs about the usefulness of self-assessment and their self-assessment ability. Therefore, the findings suggest that the accuracy of self-assessment is no different for students who hold strong beliefs and for those who hold weak beliefs about the effects of self-assessment on their learning. Taken together, the studies indicate that students on average do possess accurate self-assessment skills but only to a limited extent. The results also indicate that self-assessment is not learned through extended  experience and regular feedback, although the authors point out that other studies report differently. Similar to Kostons, van Gog & Paas (2010), the authors suggest further research should investigate if students’  self-assessment skills can be improved through formal training in self-assessment. They suggest if students have better developed self-assessment skills it is likely they will involve themselves more in effective learning and will

17

thus become better metacognitive and self-reflective learners capable of critical evaluation of their own performance— a skill highly valued in professional practice. Elder (2010) studied elementary school students’ ability to self-assess and reported that while students primarily rely on teacher feedback, developmental progression in older students shows increasing ability to use self-standards and consider multiple types of sources for their self-assessment. Likely, a combination of instructional emphasis on reflection and self-appraisal coupled with increasing developmental capacities were causal in the change. The authors cite research including the social cognitive model of self-regulated learning which emphasizes the importance of observing and evaluating one’s own behavior, and argue that it is important for children to engage in self-assessment beginning in the elementary grades. Research suggests that in elementary grades self-assessment can foster motivated, responsible learners. Students benefit from regular exposure to reflection practices as a part of their learning routine. McMillan & Hearn (2008) used Rolheiser’s (1996) four-stage implementation growth scheme to show teachers how to help students identify subsequent learning goals and strategies that can attain the goals. Initially, the teacher determines the goals and strategies and eventually students construct their own goals and strategies with teacher guidance. Teachers fully integrate self-assessment into their teaching in stages three and four, when they can give students feedback about self-assessments as well as future instructional goals and learning strategies. Rolheiser's growth scheme is useful in checking how often teachers use student self-evaluation and to determine any necessary improvements in the process. The authors argue that modifications are needed at different grade levels, but that even elementary students can understand and apply criteria to evaluate their own and others' work. For example, rather than emphasize direct instruction in helping students understand criteria, teachers can help students identify criteria by contrasting examples of good and not-so-good products. At lower levels teachers can simply provide a list of additional learning activities while higher-level students generate their own ideas about what they need to do. Student involvement in determining how to self-assess is particularly valuable. It enhances student motivation by providing a sense of ownership and responsibility. Engagement also increases intrinsic motivation to base performance more on competence and less on rewards for performance.

18

Grossman (2009) noted the difficulty many students had in demonstrating effective reflection practices. He pondered why it appeared so difficult for students to engage in metacognitive levels of reflection and offered Kegan’s theory of development as a four-step bridge for metacognition. The four steps are presenting a descriptive model of the mind, applying the model to a case, applying the model to additional cases, and metacognition (reflecting on their thoughts and feelings). The author notes that it is important for students to reach the metacognitive level of understanding to be able to report on their thoughts while solving problems and that this is especially important in mathematics, physics, and chemistry, where faculty are encouraged to have students reflect on the problem solving processes they are using.

IV. Tools for implementing SSA

Andrade, Wang, Du & Akawi (2009) studied the effects of long- and short-term use of rubrics on SSA. Students reviewed a model and used a rubric to self-assess drafts of writing. The authors collected self-efficacy ratings three times. Results showed that self-efficacy ratings for all students increased as they wrote but girls benefited the most. The authors recommend future work focus on longer treatment times and more diverse samples.

Andrade, Du & Mycek (2010) continued the 2009 focus on rubric-referenced self-assessment. This study involved using a model essay to scaffold the process of generating a list of criteria for an effective essay, reviewing a written rubric, and using the rubric to self-assess first drafts. Results showed a statistically significant positive impact on the quality of student writing as measured by essay scores. The authors suggest further research employing longer treatment time involving multiple assignments and in other domains such as in math and science where students are involved in qualitatively different kinds of work.

Andrade (2010) included Strategic Content Learning, portfolio-based assessment, and traffic lights as examples of methods in which students can be their own and their teachers’ best source of formative assessment information, helping them to self-assess and self-regulate in a supportive environment. When setting task specific goals to improve understanding the author suggests that rubrics, checklists, scoring guidelines, and detailed assignment briefs can be helpful to students.

 Fluckiger (2010) discussed the benefits of single point rubrics. She argued that the single point rubric differs from traditional multiple  point rubrics as  a tool for students to use in self-assessment to indicate four elements of learning: knowing the desired outcome, knowing the state of current understanding, knowing what improvement steps to take, and knowing

19

how to exceed the stated goal. The single point rubric, as opposed to traditional multiple point rubrics, has only has one level of performance— the proficient level. Students should be invited to create and/or revise the rubric collaboratively with the teacher. The author argues that the benefits of using single point rubrics are evidenced in the areas of student achievement, engagement in learning, and in students being effective at self-assessment.

Ibabe & Jauregizar (2010) studied the relationship between students’ frequency of use of online self-assessment with feedback and their final performance on the course, taking into account both learners’ motivation and perceived usefulness of these resources for their learning process. They also studied the relationship between metacognitive variables and academic performance and/or execution of activities aimed at learning the course content. To do so they created self-assessment material with the Hot Potatoes program and assessed the degree to which students took advantage of the tool, their satisfaction with it, and their perceived knowledge, using ad hoc questionnaires. The online Hot Potatoes program includes a variety of exercises (multiple-choice, matching exercises, short answers, incomplete sentences, and crosswords), remains accessible to students until the final exam, and can be taken as many times as students desire. Results indicate better academic performance for students who used the program and students with low motivation levels also made use of the teaching tool. The authors suggest that online self-assessment with feedback exercises can benefit learning, perhaps via metacognitive knowledge, prompting teachers to consider including self-assessment in the curriculum, since it can contribute to self-regulation, reflection, and learning.

Willey & Gardner (2010) suggested that all students can benefit from the reflective nature of self-assessment and the feedback it provides, and that these benefits can be seen as valuable and desirable so that students are eager to participate. They investigated the use of self- and peer- assessment multiple times for different purposes within a single subject. In addition, they examined whether providing students with multiple opportunities to practice and receive feedback in different contexts encouraged peer learning, increased engagement, and improved students’ desire to learn. Set in a university Engineering Design Fundamentals class, the authors used SPARK (Self and Peer Assessment Resource Kit) across four earning assessment tasks that combined to form a major design project. The results showed that the multiple use of self- and peer- assessment processes for different purposes within a single subject contributed significantly to improving the prescribed learning outcomes.  

 Donham (2010) reported that rubrics and checklists can provide effective structure for self-assessment and are more empowering when students

20

contribute to their development. By participating in the design of SSA rubrics, students gain more independence in learning how to self-assess.

Harada (2010) reported that when students conduct library research effective self-assessment tools include reflections logs, flow charts, rating scales, directed conversations, targeted writing, graphic organizers, and rubrics. The author believes that students can benefit from collaboration with teachers in establishing the rubric criteria and that such tools help students take charge of their own learning, one of the finest contributions educators can make to student success in the 21st-century world.

V. Conclusions, Implications and Future Research

a. Conclusions Over the past five years the literature includes an increasing amount of attention on SSA. Much of the research has focused on the benefits to student self-efficacy, metacognition, fostering intrinsic motivation and developing lifelong learning habits. As a relatively new field in education, significant attention has also been paid to providing operational definitions to delineate the points of differences between self-regulation, self-reflection, self-evaluation, and self-assessment. These efforts have been helpful to providing the clarity necessary for coherent and effective discourse in this promising field.

b. Implications Thanks to the collective work of the pioneers of SSA the field has reached a

point of common vocabulary and a body of early research that shows directional promise in fulfilling the purported student benefits. With the theoretical framework becoming increasingly verified and accepted across the educational community, attention now needs to be directed to action research, teacher implementation and professional practice in order to validate SSA benefits as well as add to the body of knowledge on alternate approaches.

c. Future Research

The literature currently reveals a dearth of research that takes a holistic approach to designing and implementing an effective, ongoing SSA program at the classroom level. The field may benefit by educators, represented across the spectrum of all grade levels and subject areas: 1) developing a comprehensive understanding of SSA, the research to date, and potential benefits, 2) determining SSA strategies likely to make a positive difference to student learning in their curriculum, 3) teaching students how to effectively employ SSA, 4) involving students in designing SSA tools to the maximum extent possible, 5) designing the training necessary for students to effectively use the tool(s), 6) implementing SSA using action research protocols in order to quantify the impact of change in practice, and 7) publishing the results, adding to the growing body of understanding in this promising field to maximize formal education outcomes and empower students as intrinsically motivated, lifelong learners.

21

References

Ali, R. (2009). Teacher expectations, students' motivation and self perception in private schooling. Bulletin Of Education & Research, 31(2), 45-60. Retrieved from http:// web.ebscohost.com.ezproxylocal.library.nova.edu/ehost/pdfviewer/pdfviewer?vid= 23&hid=108&sid=53676af5-04aa-49da-b242-022acbec7b0e%40sessionmgr104

Anderman, L. H., & Kaplan, A. (2008). The role of interpersonal relationships in student

motivation: Introduction to the special issue. Journal Of Experimental Education, 76(2), 115-119. doi: 10.3200/JEXE.76.2.115-120

Andrade, H. L, Du, Y., & Mycek, K. (2010). Rubric-referenced self-assessment and middle

school students’ writing. Assessment in Education: Principles, Policy & Practice, 17(2), 199-214. Retrieved from https://illiad-library-nova-edu.ezproxy.library.nova.e du/FNN/illiad.dll?SessionID=M024559038U&Action=10&Form=75&Value=665054

Andrade, H. L. (2010). Students as the definitive source of formative assessment. In H. L.

Andrade & G. J. Cizek (Eds.), Handbook of Formative Assessment (pp. 90-105). New York: Routledge.

Andrade, H. L., Wang, X., Du, Y., & Akawi, R. L. (2009). Rubric-referenced self-assessment

and self-efficacy for writing. The Journal of Educational Research, 102(4), 287-301,320. Retrieved from http://ezproxylocal.library. nova.edu/docview/204214921?accountid= 6579

Bingham, G., Holbrook, T., & Meyers, L. E. (2010). Using self-assessments in elementary

classrooms. Phi Delta Kappan, 91(5), 59-61. Retrieved from http://web.ebscohost. com.ezproxylocal.library.nova.edu/ehost/pdfviewer/pdfviewer?vid=13&hid=113&si d=16596314-28fe-420f-a05c76f40951a55c%40sessionmgr112

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in

Education, 5(1), 7-74. Retrieved from http://ezproxylocal.library.nova.edu/login?url= lhttp://search.proquest.com/docview/204052267?accountid=6579

Brady-Amoon, P., & Fuertes, J. N. (2011). Self-efficacy, self-rated abilities, adjustment, and

academic performance. Journal of Counseling & Development, 89(4), 431-438. Retrieved from http://ezproxylocal.library.nova.edu/docview/898322886?accountid=6579

 Brookhart, S. M., Andolina, M., & Zuza, M. (2004). Minute math: An action research study

of student self-assessment. Educational Studies In Mathematics, 57(2), 213-227. Retrieved from http://web.ebscohost.com.ezproxylocal.library.nova.edu/ehost/pdfviewer/ pdfviewer?vid=14&hid=113&sid=16596314-28fe-420f-a05c-76f4095a55c%40session mgr112

Carr, S. C. (2008). Student and peer evaluation: Feedback for all learners. Teaching

Exceptional Children, 40(5), 24-30. Retrieved from http://web.ebscohost.com.ez

22

proxylocal.library.nova.edu/ehost/pdfviewer/pdfviewer?vid=8&hid=108&sid=5367 af5-04aa-49da-b242-022acbec7b0e%40sessionmgr104

Cauley, K. M., & McMillan, J. H. (2010). Formative assessment techniques to support

student motivation and achievement. The Clearing House, 83(1), 1-6. Retrieved from http://ezproxylocal.library.nova.edu/docview/848217350?accountid=6579

Clauss, J., & Geedey, K. (2010). Knowledge surveys: Students ability to self-assess. Journal

of the Scholarship of Teaching and Learning, 10(2), 14-24. Retrieved from http://ez proxylocal.library.nova.edu/docview/754908840?accountid=6579

Cole, J. S., & Denzine, G. M. (2004). “I'm not doing as well in this class as I'd like to”:

Exploring achievement motivation and personality. Journal of College Reading and Learning, 34(2), 29-44. Retrieved from http://web.ebscohost.com.ezproxylocal.librar y.nova.edu/ehost/pdfviewer/pdfviewer?vid=12&hid=113&sid=16596314-28fe-420f- a05c-76f40951a55c%40sessionmgr112

Den Brok, P., Levy, J., & Brekelmans, M. (2005). The effect of teacher interpersonal

behaviour on students' subject-specific motivation. Journal Of Classroom Interaction, 40(2), 20-33. Retrieved from http://web.ebscohost.com.ezproxylocal.library.nova.edu/ ehost/pdfviewer/pdfviewer?vid=27&hid=108&sid=53676af5-04aa-49da-b242-022acb ec7b0e%40sessionmgr104

Donham, J. (2010). Creating Personal Learning through Self-Assessment. Teacher Librarian,

37(3), 14-21.. Retrieved from http://web.ebscohost.com.ezproxylocal.library.nova.ed u/ehost/pdfviewer/pdfviewer?sid=2e0d8d44-4222-4f6e-9457-5c0d99e029ed%40sessi onmgr114&vid=8&hid=106

Elder, A. D. (2010). Children’s self-assessment of their school work in elementary school.

Education 3-13, 38(1), 5-11. Retrieved from http://ezproxylocal.library.nova.edu/doc view/742868697?accountid=6579

Fluckiger, J. (2010). Single point rubric: A tool for responsible student self-assessment. Delta

Kappa Gamma Bulletin, 76(4), 18-25. Retrieved from http://web.ebscohost.com.ezpro xylocal.library.nova.edu/ehost/pdfviewer/pdfviewer?vid=13&hid=113&sid=1659631 4-28fe-420f-a05c-76f40951a55c%40sessionmgr112

Grossman, R. (2009). Structures for facilitating student reflection. College Teaching, 57(1),

15-22. Retrieved from http://web.ebscohost.com.ezproxylocal.library.nova.edu/ehost/ pdfviewer/pdfviewer?vid=13&hid=108&sid=8bfa0d4e-3453-4105-a6e4-5b2dd39cb9f

e%40sessionmgr104 Halawah, I. (2006). The effect of motivation, family environment, and student characteristics

on academic achievement. Journal Of Instructional Psychology, 33(2), 91-99. Retrieved from http://web.ebscohost.com.ezproxylocal.library.nova.edu/ehost/pdfvie wer/pdfviewer?vid=27&hid=108&sid=53676af5-04aa-49da-b242-022acbec7b0e%40

23

sessionmgr104 Harada, V. H. (2010). Self-assessment: Challenging students to take charge of learning.

School Library Monthly, 26(10), 13-15. Retrieved from http://ezproxylocal.library. nova.edu/doc view/347847627?accountid=6579

Heritage, M. (2009). Using self-assessment to chart students’ paths. Middle School

Journal, 40(5), 27-30. Retrieved from http://ezproxylocal.library.nova.edu/docview/ 217435988?accountid=6579

Hughes, G. B. (2010). Formative assessment practices that maximize learning for students at

risk. In H. L. Andrade & G. J. Cizek (Eds.), Handbook of Formative Assessment (pp. 212-232). New York: Routledge.

Ibabe, I., & Jauregizar, J. (2010). Online self-assessment with feedback and metacognitive

knowledge. Higher Education, 59(2), 243-258. doi.10.1007/s10734-009-9245-6 Kauffman, D. F., & Husman, J. (2004). Effects of time perspective on student motivation:

Introduction to a special issue. Educational Psychology Review, 16(1), 1-7. doi:10.10 23/B:EDPR.0000012342.37854.58

Kesici, S., & Erdogan, A. (2010). Mathematics anxiety according to middle school students’

achievement motivation and social comparison. Education, 131(1), 54-63. Retrieved from http://web.ebscohost.com.ezproxylocal.library.nova.edu/ehost/pdfviewer/pdfvie wer?vid=29&hid=108&sid=53676af5-04aa-49da-b242-022acbec7b0e%40sessionmgr 104

Kirby, J. R., Knapper, C., Lamon, P., & Egnatoff, W. J. (2010). Development of a scale to

measure lifelong learning. International Journal of Lifelong Education, 29(3), 291-302. doi:10.1080/02601371003700584

Kitsantas, A., Reiser, R. A., & Doster, J. (2004). Developing self-regulated learners: Goal

setting, self-evaluation, and organizational signals during acquisition of procedural skills. Journal Of Experimental Education, 72(4), 269-287. Retrieved from http://web.ebsco host.com.ezproxylocal.library.nova.edu/ehost/pdfviewer/pdfviewer?vid=14& hid=108&sid=53676af5-04aa-49da-b242-022acbec7b0e%40sessionmgr104

Kizilgunes, B., Tekkaya, C., & Sungur, S. (2009). Modeling the relations among students'

epistemological beliefs, motivation, learning approach, and achievement. Journal Of Educational Research, 102(4), 243-255. Retrieved from http://web.ebscohost.com.ez proxylocal.library.nova.edu/ehost/pdfviewer/pdfviewer?vid=29&hid=108&sid=5367a f5-04aa-49da-b242-022acbec7b0e%40sessionmgr104

Kostons, D., van Gog, T., & Paas, F. (2010). Self-assessment and task selection in learner-

24

controlled instruction: Differences between effective and ineffective learners. Computers & Education, 54(4), 932-940. Retrieved from http://ezproxylocal.library.nova.edu/doc view/61803893?accountid=6579

Lew, M. D. N., Alwis, W. A. M., & Schmidt, H. G. (2010). Accuracy of students’ self-

assessment and their beliefs about its utility. Assessment & Evaluation in Higher Education, 35(2), 135-156. Retrieved from https://illiad-library-nova-edu.ezproxy.libr ary.nova.edu/FNN/illiad.dll?SessionID=I174940751G&Action=10&Form=75&Valu e=665051

Margolis, H., & McCabe, P. P. (2004). Self-efficacy: A key to improving the motivation of

struggling learners. Clearing House, 77(6), 241-249. Retrieved from http://web.ebsco host.com.ezproxylocal.library.nova.edu/ehost/pdfviewer/pdfviewer?vid=13&hid=11 3&sid=16596314-28fe-420f-a05c-76f40951a55c%40sessionmgr112

McMillan, J. H. (2010). The practical implications of educational aims and contexts for

formative assessment. In H. L. Andrade & G. J. Cizek (Eds.), Handbook of Formative Assessment (pp. 41-58). New York: Routledge.

McMillan, J. H., & Hearn, J. (2008). Student self-assessment: The key to stronger student

motivation and higher achievement. Educational Horizons, 87(1), 40-49. Retrieved from http://web.ebscohost.com.ezproxylocal.library.nova.edu/ehost/pdfviewer/pdfviewer/pdf viewer?vid=18&hid=108&sid=53676af5-04aa-49da-b242-022acbec7b0e%40sessionmgr 104

Ozogul, G., & Sullivan, H. (2009). Student performance and attitudes under formative

evaluation by teacher, self and peer evaluators. Educational Technology Research and Development, 57(3), 393-410. Retrieved from http://ezproxylocal.library.nova.edu/do cview/61888079?accountid=6579

Ramirez, B. U. (2010). Effect of self-assessment on test scores: Student perceptions. Advances

in Physiology Education, 34(3), 134-136. Retrieved from http://ezproxylocal.library. nova.edu/docview/762466441?accountid=6579

Schweinle, A., Meyer, D. K., & Turner, J. C. (2006). Striking the right balance: Students' motivation and affect in elementary mathematics. Journal Of Educational Research, 99(5), 271-293. doi: 10.3200/JOER.99.5.271-294

Shores, M. L., & Shannon, D. M. (2007). The effects of self-regulation, motivation, anxiety,

and attributions on mathematics achievement for fifth and sixth grade students. School Science And Mathematics,107(6), 225-236. doi:10.1111/j.1949-8594.2007.tb18284.x

Wang, K. H., Wang, T. H., Wang, W. L., & Huang, S. C. (2006). Learning styles and

formative assessment strategy: Enhancing student achievement in web-based learning. Journal Of Computer Assisted Learning, 22(3), 207-217. doi:10.111 1/j.1365-2729.20 06.00166.x

25

Willey, K., & Gardner, A. (2010). Investigating the capacity of self and peer assessment

activities to engage students and promote learning. European Journal of Engineering Education, 35(4), 429-443. Retrieved from https://illiad-library-nova-edu.ezproxy.libr ary.nova.edu/FNN/illiad.dll?SessionID=I174940751G&Action=10&Form=75&Value=665273

Zimmerman, B. J. (2002). Becoming a self-regulated learner: an overview. Theory Into

Practice, 41(2), 64-70. Retrieved from http://web.ebscohost.com.ezproxylocal.library. nova.edu/ehost/pdfviewer/pdfviewer?vid=25&hid=122&sid=e1423f13-4654-480c-8e 78-bd58e6f5a1d0%40sessionmgr15