1. 4.1 4.2. . 2010/10/124.1 4.2

2. AGENDA 12 . 3. 4. .4.1 4.2 3. AGENDA 12 . 3. 4. .4.1 4.2 4. 4 12 ..4.1 4.2 5. AGENDA 12 . 3. 4. .4.1 4.2 6. (classication categorization)4.1 4.2 7. 2 SVM 4.1 4.2 8. AGENDA 12 . 3. 4 . .4.1 4.2 9. d P(c|d)cC P(c|d) 1P(c)P(d|c) P(c|d) =P(d) . 2P(d) P(c)P(d|c) c max .P(c)P(d|c) c max= arg maxcP(d)= arg max P(c)P(d|c)c. 4.14.2 10. P(d|c)d d d P(d|c)d - -4.1 4.2 11. AGENDA 12 . 3. 4 . .4.1 4.2 12. - P(d|c) d,d P(d|c) = p,c (1 p,c )1,dVVpcP(c)p,c pc2 ,d p,c (1 p,c )1,dc d 4.1 4.2 13. - ,d P(c)P(d|c) = pc p,c (1 p,c )1,d V cp,cP(d|c) P(d|c) p,c 14.1 4.2 14. D D= {(d(1) , c(1) ), (d(2) , c(2) ), ..., (d|D| , c|D| )} log P(D) =log P(d, c)(d,c)D ,d =log pc p,c (1 p,c )1,d (d,c)DV log pc + = (,d log p,c + (1 ,d ) log(1 p,c )) (d,c)D V =N c log pc + N,c log p,c + (N c N,c ) log(1 p,c ) c c Vc VNc :c N,c :c4.1 4.2 15. pc max . log P(D)s.t.pc = 1. cL(, ) L(, ) = log P(D) + pc 1 c:{ pomega,c }inV,cC , {pc } cC 4.1 4.2 16. L(, )= 0 p,c L(, )= 0 pc L(, )= 0 4.1 4.2 17. L(, ) = N c log pc +N,c log p,c p,c p,c c c V + (N c N,c ) log(1 p,c ) + pc 1 c V c(1p,c )N,cp,c=+ (N c N,c )p,c(1 p,c )N,c (N c N,c )= p,c 1 p,c L(, ) = N c log pc + N,c log p,c pc pc c c V + (N c N,c ) log(1 p,c ) + pc 1 c V cNc=+pc4.1 4.2 18. p,c N,c (N c N,c ) = 0 p,c1 p,c (1 p,c )N,c p,c (N c N,c ) = 0p,c (N c N,c + N,c ) = N,cN,cp,c = Nc4.1 4.2 19. pc Nc + =0pc Nc pc= pc =1 c 1 Nc =1 c = Nc c Nc Nc pc= = c Nc4.1 4.2 20. c p,c =ccpc =4.1 4.2 21. 4.1P3 d(1) = good bad good good d(2) = exciting exciting d(3) = good good exciting boringN3 d(4) = bad boring boring boring d(5) = bad good bad d(6) = bad bad boring exciting PN 4.1 4.2 22. 4.1V={bad, boring, exciting, good}N P = 3, N N = 3, N bad,P = 1, N bad,N = 3, N boring,P = 1, N boring,N = 2,Nexciting,P = 2,Nexciting,N = 1, N good,P = 2,N good,N = 1, NP NN pP =N P +N N= 3+3 = 0.503pN = N p+NN = 3+3 = 0.50 3 N bad,PN bad,N pbad,P = N P = 1 = 0.333pbad,N = NN = 3 = 1.00 3 N boring,PN bof ing,N pboring,P = N P = 3 = 0.331pbof ing,N = NN = 2 =30.67 Nexciting,P pexciting,P = N P = 2 = 0.67 3Nexciting,N 1 pexciting,N == 3 = 0.33 N good,PN good,N pgood,P = N P = 2 = 0.67 3 pgood,N = NN = 1 = 0.333 4.1 4.2 23. 4.24.1d d = good good bad boringpP pd|P pN pd|N pP pd|P= pP pbad,P pboring,P (1 pexciting,P ) pgood,P= 0.5 0.33 0.33 (1 0.67) 0.67 = 0.012 pN pd|N= pN pbad,N pboring,N (1 pexciting,N ) pgood,N= 0.5 1.00 0.67times(1 0.33) 0.33 = 0.0744.1 d N 4.1 4.2 24. 4.24.1d d = good good bad boringpP pd|P pN pd|N pP pd|P= pP pbad,P pboring,P (1 pexciting,P ) pgood,P= 0.5 0.33 0.33 (1 0.67) 0.67 = 0.012 pN pd|N= pN pbad,N pboring,N (1 pexciting,N ) pgood,N= 0.5 1.00 0.67times(1 0.33) 0.33 = 0.0744.1 d N 4.1 4.2 25. 4.3 4.1 d(1)d(1) = good bad good good ne d d = bad bad boring boring ne 4.1 4.2 26. 4.3 nene N f ine,P N f ine,N p f ine,P = NP = 1 3 = 0.33p f ine,N = NN = 0 3 = 0.00 pP pd|P = pP pbad,P pboring,P (1 pexciting,P ) p f ine,P (1 pgood,P ) = 0.5 0.33 0.33 (1 0.67) 0.33 (1 0.67) = 0.002 pN pd|N = pN pbad,N pboring,N (1 pexciting,N ) p f ine,N (1 pgood,N ) = 0.5 1.00 0.67 (1 0.33) 0.00 0.67 = 0.00 P4.1 4.2 27. 4.3d bad boringgood excitingP p f ine,N = 0.00NpN pd|N = 0.00 0MAP4.1 4.2 28. MAP0.00MAP 1 1 log P() + log P(D) log pc p,c + log P(d, c) c,c(d,c) D = ( 1)log pc + ( 1) log p,c c ,c 1,d + log pc ,d( p,c (1 p,c )) (d,c) DVc p(c) = 1 4.1 4.2 29. MAP L(, ) = log P() + log P( D) + pc 1 c L(, ) ( 1) N,cN c N,c=+ p,cp,c p,c1 p,c L(, ) ( 1) N c=++ pcpcpc4.1 4.2 30. MAP 0 c pc = 1N,c + ( 1) p,c = Nc + 2Nc + 1 pc = c N c + |C| 4.1 4.2 31. 4.44.3 MAP=1 P3 d(1) = good bad good good ne d(2) = exciting exciting d(3) = good good exciting boringN3 d(4) = bad boring boring boring d(5)= bad good bad d(6) = bad bad boring exciting4.1 4.2 32. 4.4Table: MAP MAPpP0.50 0.50 pN0.50 0.50pbad,P0.33 0.40 pbad,N1.00 0.80 pboring,P0.33 0.40pboring,N0.67 0.60pexciting,P 0.67 0.60 pexciting,N 0.33 0.40p f ine,P 0.33 0.40 p f ine,N 0.00 0.20pgood,P 0.67 0.60 pgood,N 0.33 0.40MAPsmoothing MAP4.14.2 33. AGENDA 12 . 3. 4 . .4.1 4.2 34. V1|d|P(d|c) d n,d ( n )! ,d P(d|c) = P K = n,d n,d q,cV n,d ! V K:( ) P K = n,d : n,d 4.1 4.2 35. c ( n )! ,dpc P n,d n,dP(c)P(d|c) = q,c V n,d ! V ( n )! ,d arg max P(c)P(d|c) = arg max pc P n,d n q ,dcc V n,d ! V ,c n= arg max pc q,c c V nc pc V q,c4.1 4.2 36. 2 4.1 4.2 37. log P( D) =log P(d, c) (d,c) D p(|d|)|d|! n ,d = log pcq,c (d,c) DVn !,d VP(|d|)|d|! = log +log pc +n,d log q,c (d,c) D V n,d ! (d,c) D(d,c) D V P(|d|)|d|! = log +log nc pc +n,c log q,c (d,c) D V n,d ! c c V max. log P(D) s.t. pc = 1. cC q,c = 1; c C V4.1 4.2 38. L(, , ) = log P(D) + c q,c 1 + pc 1 cCV cC L(, , )= 0 q,c L(, , )= 0 pc L(, , )= 0 L(, , )= 0 4.1 4.2 39. L(, , ) P(|d|)|d|! = log + nc log pc +n,c log q,c q,c q,c(d,c)D V n,d ! c c V c (1) + (pc 1) cC VcC n,c =+ c = 0 q,c n,cq,c = c4.1 4.2 40. cq,c = 1V1 n,c = 1 c V1 c = Vn,cn,cq,c = n,cpc4.1 4.2 41. c q,c =c c p,c =c4.1 4.2 42. MAP0.00MAPMAP 1 log P() + log P(D) log p1 q,c + c log P(d, c) c,c (d,c)D P(|d|)|d|! n ,d = ( 1) log pc + log q,c + log pcq,c n ! c,c (d,c)D V ,d V c p(c) = 1 q,c = 14.1 4.2 43. MAPL(, , ) = log P() + log P(D) + c p,c 1 + pc 1 cC V cC L(, , ) ( 1) n,c =++ c q,c q,c q,c 0V q,c = 1n,c + ( 1) q,c = n,c + |W|( 1) 4.1 4.2 44. AGENDA 12 . 3. 4 . .4.1 4.2 45. d MAP 4.1 4.2 46. ( ) Ml for nlp chapter 44.1 4.2 47. 4.1 4.2