+

Logics and Ontologies for Portuguese Understanding

Valeria de Paiva, Nuance Comms, CA Visiting Prof DI-PUC-RJ, CSF (with Alexandre Rademaker, Gerard de Melo, Claudia Freitas, Livy Real, Dario Oliveira,Suemi Higuchi …)

+Nuance Comms, AI and NL Lab, Sunnyvale, CA

+Ron Kaplan “Beyond the GUI: It’s Time for a Conversational User Interface”, Wired 2013

http://www.wired.com/2013/03/conversational-user-interface/

+Siri and other personal assistants

+The Future is Meaning…

+Recent Past…

+PARC’s Bridge System (1999-2008)

+ PARC’s Bridge System (1999-2008)

F-structure semantics

KR

Parsing KR Mapping

Inference Engines Text

Sources

Question

Assertions

Query

Grammar Stanford Parser

Textual Inference logics

Term rewriting OpenWN-PT SUMO-PT KR mapping rules

+Powerset

Acquired by Microsoft, 2008

+and Cuil…

+Another story

https://www.parc.com/event/934/adventures-in-searchland.html

+Today: redoing PARC work in Portuguese…

+Goals in 2010

+Goals in 2010

+Goals in 2010…

¨  Content analysis ⁄ large-scale intelligent information extraction, access and retrieval ¨  Text understanding ¨  Text generation ¨  Text simplification ¨  Automatic summarization ¨  Dialogue systems ¨  Question answering ¨  Machine Translation ¨  Named Entity Recognition, ¨  Anaphora/co-reference resolution, ¨  Reading, writing, grammar aids, etc...

+Pipeline envisaged in 2010 Plug and Play…

+SRI Talk: www.slideshare.net/valeria.depaiva/bridge-sri

+ PARC’s Bridge System (1999-2008)

Idea: Simplify and reproduce components in PORTUGUESE

F-structure semantics

KR

Parsing KR Mapping

Inference Engines Text

Sources

Question

Assertions

Query

Grammar Stanford Parser

Textual Inference logics

Term rewriting OpenWN-PT SUMO-PT KR mapping rules

+Reality Check…

n  What PARC considered pre-processing is MOST of the processing…

n  Got the XLE research license, but hard to use it, needed several lexicons that DO NOT exist in Portuguese, notably WordNet

n  There are several open toolkits that can be used instead: FREELING

OpenNLTK

StanfordNLP

More usable, more community, less expertise required

+Why do we need lexical resources?

n  Sem eles nao fazemos nada!

n  Precisamos de dicionarios, gazetteers, tesauros, lexica de todos os tipos

n  Queremos um sistema que ENTENDE e RACIOCINA em PORTUGUES

n  MAS pra fazer logica e raciocinio com representacoes de conhecimento vindo de textos, de maneira escalavel e robusta, temos que lidar com linguagem seriamente…

n  QUAIS recursos lexicos? Comecamos com WordNet e NomLex…

+Since 2010…

n Dois recursos lexicos grandes e importantes: OpenWN-PT e NomLex-PT

n Grupo cresceu e varias agendas de pesquisa estao misturadas

n Hoje: uma tentativa de organizar o projeto mais ambicioso, no estilo do de 2010.

+Portuguese WordNet?

n OpenWordNet-PT our big success so far…

n  Used by FreeLing as their internal dictionary for Portuguese

n  Used by Onto.PT for calibration..

n  Most importantly used by Bond in Open Multilingual WordNet http://compling.hss.ntu.edu.sg/omw/

n  Used by other researchers (anedoctal evidence)

+WordNet?

http://wordnetweb.princeton.edu/

+WordNet…

n  Is a combination of dictionary and thesaurus that is intuitive, usable, and supports automatic text analysis and artificial intelligence applications. Released under a BSD style license, can be downloaded and used freely.

n WordNet is the most commonly used computational lexicon of English.

n  WordNet created at Princeton University under George A. Miller, since 1985. A lexical database for English: groups English words into sets of synonyms called synsets, provides short, general definitions, and records the various semantic relations between these

n  [ Some complaints that WordNet encodes sense distinctions that are too fine-grained even for humans. The granularity issue has been tackled by proposing clustering methods that automatically group together similar senses of the same word.]

+Global WordNet Association n  Christiane Fellbaum and Piek Vossen

(EuroWordNet 1996-1999, GWA since)

n  The Global WordNet Association (GWA) is a free, public and non-commercial organization that provides a platform for discussing, sharing and connecting wordnets for all languages in the world.

n  Open Multilingual Wordnet Francis Bond http://compling.hss.ntu.edu.sg/omw

n  A simple user interface: Welcome to the Open Multi-lingual Wordnet (1.2) http://compling.hss.ntu.edu.sg/omw/cgi-bin/wn-gridx.cgi?gridmode=grid

+Multilingual Wordnet 1.0

+Multilingual Wordnet 1.2

data for over 150 languages, with an estimated accuracy of 94%....

+Multilingual Wordnet 1.2

+ OpenWordnet-PT? (aren’t all wordnets open?)

We needed a Portuguese Wordnet for our work, but none of the previous projects was openly available.

Previous work: WordNet.PT and WordNet.PT global (Lisboa), MultiWordNet.PT and Brazilian WordNet by Bento Dias.

+Portuguese WordNets… n  WordNet.PT (P. Marrafa) since 1999, part

of EuroWordNet, 19K expressions, manually curated, online consulting only. Some domains

n  MWN.PT - MultiWordnet of Portuguese (A. Branco), since 2008, part of MWN, over 17,200 manually validated concepts/synsets, not free

n  WN.Br (B. Dias da Silva) since 2000, not open, not available online, REBECA 2010 only ‘wheeled vehicles’….

n  SIMULTANEOUS to ours: Onto.PT, Coimbra http://ontopt.dei.uc.pt/ Hugo Gonçalo Oliveira e Paulo Gomes

+ OpenWN-PT:What?

n  Leverage EuroWordNet, MultiWordNet, Global WordNet experience

n  Gerard de Melo joined project, leverage YAGO, UWN/Menta experience…

n  UWN/MENTA (de Melo/Weikum) A large-scale multilingual lexical knowledge base built using statistical methods, transforming WordNet into a massively multilingual resource (over 1 million words and several million named entities in a single large multilingual taxonomy)

n  Portuguese `projection’ of UWN/Menta the basis of automated version of OpenWordNet-PT, publicly available.

https://github.com/arademaker/wordnet-br

+OpenWN-PT: the method n  a two-tiered methodology: high precision for more

frequent words, high recall to cover the long tail

n  Translation dictionaries to map the English members of a synset to possible Portuguese translation candidates

n  To disambiguate and choose the correct translations, feature vectors for possible translations are created by computing graph-based statistics in the graph of words, translations, and synsets.

n  Monolingual wordnets and parallel corpora used to enrich this graph. Statistical learning techniques used to iteratively refine this information and build an output graph connecting Portuguese words to synsets.

n  Wikipedia pages are then linked to relevant WordNet synsets by learning from similar graph-based features as well as gloss similarity scores.

+OpenWN-PT: some problems…

Capitalized items, plurals, duplicates, a few gender issues, missing items…

+OpenWN-PT: what’s missing? how to correct it?

n  Correct items one by one as applications require, doesn’t work. We tried it.

n  Need broad phenomena to be dealt with

n  Need context to understand synsets, both in English and in Portuguese

n  So far concentrate in the phenomenon of NOMINALIZATION (DEVERBALS), Livy Real’s thesis, Valeria’s previous work with Olya Gurevich

n  Hence two resources OpenWordNet-PT and NomLex-PT

+OpenWN-PT: RDF Representation

n  Why? Forinteroperability between wordnets. To be able to rely on Linked Data and Semantic Web standards such as RDF and OWL.

n  Both the data model and the actual data in the same format. Range of existing data processing tools, including databases (“triple stores”) with SQL-like query interfaces (SPARQL).

n  Standard W3C encoding of WordNet in RDF since 2006. OpenWN-PT is is modelled after and fully interoperable with Princeton WordNet.

n  This means that one can easily find Portuguese equivalents for specific English word senses and vice versa. Also means OpenWN-PT is part of a large ecosystem of compatible resources, including domain identifiers and mappings to Wikipedia.

+Use Cases: FreeLing

n  Word Sense Disambiguation via FreeLing 3.0 An Open Source Suite of Language Analyzers

n  OpenWN-PT has been incorporated into FreeLing (Padro’ and Stanilovsky, 2012)

n  Using Freeling’s word sense disambiguation framework, a given Portuguese text can automatically be annotated with word senses.

n  UPC, Barcelona

+Use Cases: Sentiment Analysis

n  Sentiment Analysis, using tweets about soccer games

n  OpenWN-PT and SentiWordNet to compare the MachineLearning-based sentiment analysis integrated into IBM InfoSphere Streams (ISS) platform.

n  1 million tweets, 4 friendly matches Brazilian team in 2013, 7 classes of positivity

n  IBM Research, BR

+Use Cases: NomLex-PT (Livy Real)

n  extension of OpenWN-PT aims at incorporating links to connect deverbal nouns with their corresponding verbs.

n  For English, NOMLEX (Macleod et al., 1998) has provided extensive descriptions of nominalizations via extensions of initial core.

n  NOMLEX-BR Translation of initial core, plus French Nomage

n  Overall, we initially created over 2,000 entries. These have been integrated into OpenWN-PT.

n  Hope to facilitate the use of nominalizations for linguistic research as well as information extraction

The destruction of the city by Alexander in 330BC…

+Use Cases: NomLex-PT (Livy Real)

n  Incorporating NOMLEX-PT data into OpenWN-PT has shown itself useful in pinpointing some issues with the coherence and richness of OpenWN-PT.

n  the word abasement corresponds in NOMLEX to the verb abase, and thus we would like a similar correspondence between the Portuguese noun aviltamento and the verb aviltar (our suggested translations). OpenWN-PT simply has two synsets humilhar, abaixar and humilhar, rebaixar. The more common verb humilhar is repeated, while the uncommon aviltar was left out.

n  Other useful kinds of relationships between parts of speech (say the connections between adjectives and adverbs) are likely to also help to improve the accuracy and richness

+Use Cases: NomLex-PT (Claudia Freitas)

n  Given that NOMLEX-PT has been created via data in lexica: NomLex, Nomage, Wicktionary, etc.. Maybe should use corpora to check this data?

n  Add all the nominalizations with suffixes chosen that were in the corpus, but not in NomLex-PT.

n  Classify these as AGENTIVE or NOT, or LEXICALIZED or NOT, or BOTH—lexicalized and not.

n  Examples: procuracao. (LEX), declaracao (BOTH)

+Main Application: DHBB

n  Dictionario Historico de Biografias Brasileiro building a domain-specific ontology using NLP

n  Paper (next month) on Digital Humanities (Dario and Suemi CPDOC, FGV), preliminary results

n  Need serious NER (Named Entity Recognition)

n  Various systems: FreeLing, Stanford, NLTK, OpenNLP….

+More Experiments: Verb Lexicon

n  Restrict to the verbs, what can we say about OpenWN-PT?

n  TorporES paper

n  VerbOcean comparison

n  What about language reasoning? Factives and Implicatives, CICLing paper 2014

n  What about the real world? SUMO in Portuguese in the plans

+ Conclusions n  Discussed the implementation and some

applications of OpenWordNet-PT, an open Word- Net for Brazilian Portuguese.

n  Cleaning up, better coverage and nominalization links connecting nouns and verbs. (9 papers in 2014..)

n  The resource has been used in developing a high-throughput commercial system as well as in a cultural heritage project, and we anticipate that numerous further applications will follow.

n  The data is freely available from https://github.com/arademaker/openWordnet-PT and a SPARQL Endpoint at logics.emap.fgv.br:10035.

n  Browsing via Open Multilingual Wordnet is fun http://compling.hss.ntu.edu.sg/omw/cgi-bin/wn-gridx.cgi

+ OpenWN-PT: next steps?..

n  First finish translating the “core” synsets in the Princeton WordNet to Portuguese?

n  Glosses? From Onto.PT?

n  Increase number of relations in OpenWN-PT as a way of improving adequacy and coherence?

n  Adding the Portuguese terms that satisfy different relations?

n  OpenVerbNet-PT?

n  Since we have a first target corpus, the Brazilian Historical Biographic Dictionary, we can also calculate word frequency to prioritize expansion of the OpenWN-PT and MORE ontology building...

n OpenBridge-PT? umbrella term for all possible improvements?

+OntoLog 2014

+

Thanks!

+References Revisiting a Brazilian Wordnet. Valeria de Paiva, Alexandre Rademaker,  (2012) Proceedings of Global Wordnet Conference, Global Wordnet Association, Matsue. OpenWordNet-PT: An Open Brazilian WordNet For Reasoning. de Paiva, Valeria, Alexandre Rademaker, and Gerard de Melo. In Proceedings of the 24th International Conference On Computational Linguistics. http://hdl.handle.net/10438/10274. OpenWordNet-PT: A Project Report. Alexandre Rademaker, Valeria de Paiva, Gerard de Melo, Livy Real and Maira Gatti. Proceedings of the 7th Global Wordnet Conference, Tartu, Estonia. Global Wordnet Association, 2014. Embedding NomLex-BR Nominalizations Into OpenWordnet-PT. Coelho, Livy Maria Real, Alexandre Rademaker, Valeria De Paiva, and Gerard de Melo. 2014. In Proceedings of the 7th Global WordNet Conference. Tartu, Estonia

+References Towards a Universal Wordnet by Learning from Combined Evidence  Gerard de Melo, Gerhard Weikum (2009) 18th ACM Conference on Information and Knowledge Management (CIKM 2009), Hong Kong, China. Bridges from Language to Logic:  Concepts, Contexts and Ontologies Valeria de Paiva (2010)Logical and Semantic Frameworks with Applications, LSFA'10, Natal, Brazil, 2010. `A Basic Logic for Textual inference", AAAI Workshop on Inference for Textual Question Answering, 2005. ``Textual Inference Logic: Take Two", CONTEXT 2007. ``Precision-focused Textual Inference", Workshop on Textual Entailment and Paraphrasing, 2007. PARC's Bridge and Question Answering System Proceedings of Grammar Engineering Across Frameworks, 2007.

+Progress Report n  Checking is much easier than starting from scratch..

n  But long and tedious work to check even the initial 5k synsets suggested by GWA (not done, yet!), let alone all synsets in OpenWN-PT

n  Necessary? YES! Lexical gaps of all sorts

n  But resource is being used, warts and all…

n  Improving the resource: new data from Bond/Foster and some manual additions

+Other stuff to add in?…

n  Onto.PT, ES wordnet?

n  Editing interfaces?

n  BabelNet?

n  NER issues?

n  Temporal issues?

n  Work with grammar?

n  Work on implicatives/factives in Portuguese?

n  FOIS workshop