Information Theory for Data Management

Large-Scale Copy DetectionXin Luna DongDivesh Srivastava11OutlineMotivationWhy does copy detection matter?Examples of copying, not copying

Copy detectionIn documentsIn softwareIn databases

Summary2Why Does Copy Detection Matter? 3

Protecting rights of data providersWhy Does Copy Detection Matter? 4Detecting plagiarism in reviews, ratings

Why Does Copy Detection Matter? We ourselves use copy-paste-modify very frequentlyExtensively used in the preparation of these slides Changes to a copy consistently propagate to other copies

Copy from one, it's plagiarism. Copy from two, it's research.paraphrasing playright Wilson Miznerhttp://en.wikipedia.org/wiki/Wilson_Mizner http://quotationsbook.com/quote/30426/

Focus of this tutorial: documents, software, databasesExclude images, audio, video 5Plagiarism Detection in TestsPlagiarized essays or portions of essaysCopy detection in documents

Plagiarized programming assignmentsCopy detection in software

Plagiarized answers to factual questionsCopy detection in databases6Copying in Documents7President Bush said Thursday that his administration would not deal with Hamas, the militant group that scored a decisive victory in this weeks Palestinian elections, if it continues to pursue the destruction of Israel.President Bush said on Thursday that his administration would not deal with Hamas, the militant group that scored a decisive victory in this weeks Palestinian elections, if it continues to pursue Israels destruction.7Copying in Documents8President Bush said Thursday that his administration would not deal with Hamas, the militant group that scored a decisive victory in this weeks Palestinian elections, if it continues to pursue the destruction of Israel.President Bush said on Thursday that his administration would not deal with Hamas, the militant group that scored a decisive victory in this weeks Palestinian elections, if it continues to pursue Israels destruction.Near-duplicate of original documentMinor edits to the original documentComparison of document checksums is inadequateAt one end of the similarity spectrum8Copying in Documents9President Bush said Thursday that his administration would not deal with Hamas, the militant group that scored a decisive victory in this weeks Palestinian elections, if it continues to pursue the destruction of Israel.The landslide victory by the militant group Hamas in this weeks Palestinian elections threatens President Bushs quest for peace in the Middle East and underscores the perils of his push for democracy.9Copying in Documents10President Bush said Thursday that his administration would not deal with Hamas, the militant group that scored a decisive victory in this weeks Palestinian elections, if it continues to pursue the destruction of Israel.XThe landslide victory by the militant group Hamas in this weeks Palestinian elections threatens President Bushs quest for peace in the Middle East and underscores the perils of his push for democracy.Topical similarityNot a good answer for copy detectionFine answer for IR style queryAt other end of similarity spectrum10Copying in Documents11President Bush said Thursday that his administration would not deal with Hamas, the militant group that scored a decisive victory in this weeks Palestinian elections, if it continues to pursue the destruction of Israel.President Bush said Thursday that his United States will not deal with Hamas until it renounces its aim to destroy Israel, and reflected on the meaning of Wednesdays Palestinian elections.11Copying in Documents12President Bush said Thursday that his administration would not deal with Hamas, the militant group that scored a decisive victory in this weeks Palestinian elections, if it continues to pursue the destruction of Israel.?President Bush said Thursday that his United States will not deal with Hamas until it renounces its aim to destroy Israel, and reflected on the meaning of Wednesdays Palestinian elections.Text reuseRestatement of original document with reformulations, additionsSomewhere in the middle range of the similarity spectrum12Copying in Software13void sumProd(int n) {float sum = 0.0;float prod = 1.0;for (int i = 1; i