How MyPsychLab scores your auto-graded writing assignments

The ten MyPsychLab writing assignments are presented in class with a video and a situation to write about. Student groups, and your instructor, will work in class to help you organize your ideas and develop your written response. After class, you write your response in 150 words or more and click “Get Essay Feedback”. The software then gives you feedback to make your essay better. You are free to adjust your essay and, when you are happy with it, click, “FINISH: Submit for grading” Within 8 – 10 seconds you will receive a score. If you are still not happy with your score, you may repeat the process and resubmit your essay a second time for grading, and receive the higher of the two scores. You will receive feedback on each of your submissions to improve your score.

The scoring system: The writing assessment engine employs latent semantic analysis, a data-based

computational system for lexical content. The database for the prompts in MyPsychLabconsists of hundreds of real student essays written to the prompts and scored by experthuman scorers. MyPsychLab evaluates new essays written to the prompts on the basis ofthese data. Thus, the system is anchored in the real-world teaching and learning ofcollege psychology students across the country.

Each writing prompt has a corresponding scoring rubric that specifies the criteria on which essays are evaluated. The system applies a score of 1 through 4 on each of fiveessay traits. The traits are: Development of Ideas, Focus and Coherence, Organization,Mechanics, and Voice. Historical content falls under the Development of Ideas trait.The Focus and Coherence trait evaluates how effectively the essay argues and illustratesideas. The other three traits are writing traits. MyPsychLab calculates an overall scoreby taking a weighted average of the trait scores. The weights are:Development of Ideas: 50%Focus and Coherence: 20%Organization: 10%Mechanics: 10%Voice: 10%

Apart from the scoring, essays receive two forms of feedback. Students may viewcategorical feedback on the trait scores for their essays. This feedback describes thegeneral criteria that apply to the trait score. Students may also view constructivefeedback designed to help them write more effective essays.

Typing your response:

When you log into MyPsychLab to type your response, you will see the screen below.

There is an introductory paragraph, a paragraph discussing your response and a link for more information on how your response will be scored.After you view the video again and are ready to give a response, click “Start”.

You are then directed to the response screen below.

You may type your answer directly into the space, or compose it is a Word document to check spelling and grammar, then copy and paste it into the response space.

After submitting your response, you will receive your score as well astrait-specific feedback within 8-10 seconds. This feedback can help youimprove your response. Once you are happy with your work, click

“Finish: Submit for Grading” in the top right hand corner, and results flow tothe instructor gradebook.

After submitting your response, you will receive your score as well astrait-specific feedback within 8-10 seconds. This feedback can help youimprove your response. Once you are happy with your work, click“Finish: Submit for Grading” in the top right hand corner, and results flow tothe instructor gradebook.

Feedback

Questions about the automated grading:

Q. How does the MyPsychLab automatic essay grader work?The Pearson MyPsychLab auto-graded writing assessments use the Knowledge AnalysisTechnologies (KAT) engine, a patented technology based on over twenty years ofresearch and development. The KAT engine is based on the mathematical approachknown as Latent Semantic Analysis (LSA), which provides a sophisticated computeranalysis of text. The program assesses the total content of an essay as well as thecorrelation between the essay's content and that of training essays previously scored byexpert human readers. MyPsychLab assigns a score to each essay based in part on thesimilarity of the content of the essay to that of the training essays.

Q. What is the KAT™ engine?The KAT engine evaluates the meaning of text by examining whole passages. The KATengine is based on Pearson's unique implementation of Latent Semantic Analysis, anapproach that infers semantic similarity of words and passages by analyzing large bodiesof relevant text. LSA can then understand the meaning of text in much the same way as ahuman reader.

Q. Does the MyPsychLab automatic essay grader use keywords in its evaluation?A common misconception is that the MyPsychLab automatic essay grader scores byrecognizing keywords. It does not. If two or more different terms accurately describe aconcept, The MyPsychLab automatic essay grader will recognize either or both and scorethem equally as well. For example, The MyPsychLab automatic essay grader measures:"Understanding is very important when you reading something you want to know about"as very similar in meaning to "Comprehending is essential" even though there are noliteral words in common. Keyword based similarity would find nothing in commonbetween these two sentences.

Q. What research has been conducted on the Pearson MyPsychLab auto-gradedwriting assessments? Pearson has conducted extensive research on the reliability andvalidity of the essay component of the MyPsychLab writing assessments. The correlationbetween the scoring of this component and human graders has been shown to be as highor higher than that between two independent human graders in dozens of studies withover 200 prompts of every type. The correlation and agreement rates of the scoresproduced by the MyPsychLab automatic essay grader are better the more expert andreliable the human scores. Each of the MyPsychLab writing prompts have been testedand trained using over 400 student essays.

Q. Are students required to type their responses directly into MyPsychLab ?Students may type their responses directly into MyPsychLab if they choose; however,many students choose to compose their responses in a word-processing program and thencopy and paste them into the MyPsychLab interface.

Q. How long does it take to score a response?Most responses are scored within 8 - 10 seconds. Keep in mind that a slow or busyInternet connection may delay the scoring.

Q. How does the MyPsychLab auto-grader recognize a good essay?The auto-grader, using the KAT engine, assesses the content of an essay, as well as moremechanical aspects of writing. When a student submits an essay for scoring, the systemimmediately measures the meaning of the essay. It then compares the essay to thetraining essays, looking for similarities and assigns a holistic score in part by placing theessay in a category with the most similar training essays. Analytic scoring occurs in muchthe same way. For each trait, the system assesses that trait in the student essay, comparesit to the training essays, and then categorizes the trait in question.

Q. How does the MyPsychLab auto-grader scoring compare to the way instructorsgrade writing?The MyPsychLab auto-grader approach mirrors the way that instructors grade essays. Forexample, when instructors evaluate a student's essay, they look for characteristics thatidentify an essay as an A or C paper. Their expectations are likely based on their previousexperience as a grader and on criteria for the assignment in question. In other words,instructors search for a match between the essay itself and the criteria for a particulargrade or score. The Essay Component is trained to mimic this process.

Q. How does the MyPsychLab auto-grader score essays with highly unusual writingstyles?An essay with a highly unique writing style or unusual construction may receive anadvisory message along with a score. If an essay is off-topic, written in a language otherthan English, too brief or too repetitive, a written refusal to write, or otherwiseincomprehensible, a student will receive an advisory that his or her essay can not bescored. These advisory messages ask the student to discuss the essay and all feedback