Cantonese spellcheck with Metaphones
17
CANTONESE SPELL CHECKING WITH METAPHONES 廣東話近音/錯別字更正 full study on http://www.corgitoergosum.net/studies/ Tuesday, 14 June 2011
-
Upload
kenshin03 -
Category
Technology
-
view
492 -
download
1
description
full study on http://www.corgitoergosum.net/studies/
Transcript of Cantonese spellcheck with Metaphones
CANTONESE SPELL CHECKING WITH METAPHONES
廣東話近音/錯別字更正
full study on http://www.corgitoergosum.net/studies/
Tuesday, 14 June 2011
a study to see if this sort of spell suggestion is possible:
query: 六頂記Do you mean“鹿鼎記”?query: 六頂寄
query: 六頂你
Tuesday, 14 June 2011
Unlike more formal pingyin methods, the “HK Govt Cantonese pingyin”, commonly used in street names and identification papers in HK, is based on English pronunciation...
Tuesday, 14 June 2011
In other words, two Cantonese words should have cantonese pingyins that sound similar in English
馮曦妤
馮希如“fung hei yue”
Tuesday, 14 June 2011
query: “六頂記”
Tuesday, 14 June 2011
query: “六頂記”
pronunciation: “luk ding gei”
look up pronunciation table
Tuesday, 14 June 2011
query: “六頂記”
pronunciation: “luk ding gei”
look up pronunciation table
search phrase dictionary for similar sounding words
suggestion: Do you mean“鹿鼎記”?
Tuesday, 14 June 2011
pronunciation table
pingyin char
ban 品
ban 稟
ban 奔
ban 賓
ban 笨
ban 繽
Tuesday, 14 June 2011
using metaphone algorithm to compute “sound hashes”
陳奕迅 (chan yik suen) XNYKXN
陳亦迅 XNYKXN
陳亦信 XNYKSN
陳力神 XNLKSN
陳意順 XNYXN
陳昇 XNSNK
Tuesday, 14 June 2011
finding similar sounding words: the levenshtein distance algorithm
distance from XNYKXN
陳綺貞 XNYJNK 3
陳亦信 XNYKSN 1
陳力神 XNLKSN 2
陳意順 XNY_XN 1
陳昇 XN_SNK 4
陳日君 XNYTKWN 2
尋日送 XMYTSNK 4
粵音輸入法 YTYMSYPFT 8
Tuesday, 14 June 2011
seems like one is the safest from this limited test set
distance from XNYKXN
陳綺貞 XNYJNK 3
陳亦信 XNYKSN 1
陳力神 XNLKSN 2
陳意順 XNY_XN 1
陳昇 XN_SNK 4
陳日君 XNYTKWN 2
尋日送 XMYTSNK 4
粵音輸入法 YTYMSYPFT 8
Tuesday, 14 June 2011
some test results
query suggestion
陳亦迅 陳奕迅
馮希如 馮曦妤
蘇斯乞柴 蘇斯黑柴
馮家強 no matches
洩海皮 薛凱琪
攝海皮 no matches
攝海琪 薛凱琪
舌海肥 薛凱琪
Tuesday, 14 June 2011
some test results
query suggestion
舌海皮 薛凱琪
舌袋肥 no matches
六頂記 鹿鼎記
豬肝補氣 珠光寶氣
青山代碼 青山黛瑪
金庸海兆 金融海嘯
神探加你樂 神探伽俐略
神探加理略 神探伽俐略
Tuesday, 14 June 2011
some test results
query suggestion
深水步 深水埗
則魚涌 鰂魚涌
則魚衝 鰂魚涌
銅羅灣 銅鑼灣
以遍蓋全 以偏概全
鞠躬盡萃 鞠躬盡瘁
鬼釡神工 鬼斧神工
Tuesday, 14 June 2011
limitations
no tonal info
HK Govt pingyin is very loose
doesn’t consider similar shaped characters, like:
秦國兵變, 碧威, 獨狐求敗, 梁洛拖, 拖舟......
Tuesday, 14 June 2011
improvement areas
speed
use libyell dict both for segmentation and common words (n-grams) checking
only looks at 聲母 韻母 of first character now
english is ignored
simplified chinese rewrite
Tuesday, 14 June 2011
feedback?
Tuesday, 14 June 2011
