Detecting, Measuring and Representing Vagueness in Ontologies

1 l April 10, 20231 l

Detecting, Analyzing and Representing Vagueness in Ontologies for Facilitating Data Reuse

Dr. Panos AlexopoulosSemantic Applications Research ManageriSOCO S.A.

University of Aberdeen1/4/2014

What will I talk about

Research Questions and (some) Answers

Ongoing & Future Work

Vagueness and Ontologies

Vagueness

„Vagueness is a semantic phenomenon where predicates admit borderline cases, namely cases where it is not determinately true that the predicate applies or not“

[Shapiro, 2006]

This happens when predicates have blurred boundaries:

•What’s the threshold number of years separating old and not old films?

•What are the exact criteria that distinguish modern restaurants from non-modern?

Where does vagueness appear in an ontology?

●A class is vague if it admits borderline cases, i.e. if there are (or could be) individuals for which it is indeterminate whether they instantiate the class.

●“TallPerson”, “StrategicClient”

Vague Classes

●Datatypes whose value range consists of a set of vague terms.

●E.g. “RestaurantPriceRange” when this comprises the terms “cheap”, “moderate” and “expensive”.

Vague Datatypes

●A relation is vague if there are (or could be) pairs of individuals for which it is indeterminate whether they stand in the relation.

●“isExpertInSubject”, “belongsToGenre”

Vague Relations

Vagueness in human communication

I am telling you this is a strategic client for the firm with large-budget projects!

Come on, $300,000 in two years is hardly a

large budget!

When is vagueness in an ontology a problem?

● Defining instances: Vagueness will cause disagreement among experts when defining class and relation instances.

● Utilizing Vague Facts in Ontology-Based Systems: Reasoning results might not meet users’ expectations.

● Integrating Vague Semantic Information: The merging of particular vague elements can lead to data that will not be valid for all its users.

● Reusing Vague Datasets: The intended meaning of the dataset’s vague elements might not be compatible to the one needed in a particular application context.

Research Questions

Can we automatically detect vague ontological definitions?

Vagueness Detection

Vagueness Measurement

Vagueness Explicitness

Can we quantify the amount and importance of vagueness in an ontology and/or a dataset?

Can we make the intended meaning of vague elements in ontologies more explicit than it typically is?

Q1: Vagueness Detection

● Can we automatically determine whether an ontology entity (class, relation etc.) is vague or not?

● For example, “StrategicClient” as “A client that has a high value for the company” is vague!

● “AmericanCompany” as “A company that has legal status in the Unites States” is not!

Problem Definition


● We train a supervised classifier that may distinguish between vague and non-vague word senses, using corresponding examples from Wordnet.

● We use this classifier to determine whether a given ontology element definition is vague or not.

Approach


● 2,000 adjective senses from WordNet. ● 1,000 vague● 1,000 non-vague

● Inter-agreement of vague/non-vague annotation among 3 human judges was 0.64 (Cohen’s Kappa)

Data

Vague Senses Non Vague Senses

• Abnormal: not normal, not typical or usual or

regularor conforming to a norm

• Compound: composed of more than one part

• Impenitent: impervious to moral persuasion • Biweekly: occurring every two weeks.

• Notorious: known widely and usually

unfavorably

• Irregular: falling below the manufacturer's

standard

• Aroused: emotionally aroused • Outermost: situated at the farthest possible

point from a center.


● 80% of the data used to train a multinomial Naive Bayes classifier.

● We removed stop words and we used the bag of words assumption to represent each instance.

● The remaining 20% of the data was used as a test set.

● Classification accuracy was 84%!

Training & Evaluation


● We also used a subjective sense classifier to classify the dataset’s senses as subjective or objective.

● From the 1000 vague senses, only 167 were classified as subjective while from the 1000 non-vague ones 993.

● This shows that treating vagueness in the same way as subjectiveness is not really effective.

Comparison with Subjectivity Analyzer


● As an ontology use case we considered CiTO, an ontology that enables characterization of the nature or type of citations.

● CiTO consists primarily of relations, many of which are vague (e.g., plagiarizes).

● We selected 44 relations and we had 3 human judges manually classify them as vague or not.

● Then we applied the Wordnet-trained vagueness classifier on the textual definitions of the same relations.

Detecting Vagueness in CiTO Ontology

Q1: Vagueness DetectionDetecting Vagueness in CiTO Ontology

Vague Relations Non Vague Relations

• plagiarizes: A property indicating

that the author of the citing entity

plagiarizes the cited entity, by

including textual or other elements

from the cited entity without formal

acknowledgement of their source

• sharesAuthorInstitutionWith:

Each entity has at least one author

that shares a common institutional

affiliation with an author of the other

entity

• citesAsAuthority: The citing entity

cites the cited entity as one that

provides an authoritative

description or definition of the

subject under discussion.

• providesDataFor: The cited entity

presents data that are used in work

described in the citing entity.


● Classification Results:● 82% relations were correctly classified as vague/non-vague ● 94% accuracy for non-vague relations.● 74% accuracy for vague relations.

● Again, we classified the same relations with the subjectivity classifier:● 40% of vague/non-vague relations were classified as

subjective/objective respectively.● 94% of non-vague were classified as objective.● 7% of vague relations were classified as subjective.

Detecting Vagueness in CiTO Ontology

Q2: Vagueness Measurement

● Can we use metrics to quantify the existence and importance of vagueness in ontologies and semantic data?

● E.g. can we somehow claim that ontology A is more vague than ontology B and have a number to support this claim?

● Some initial metrics we have thought:

● Vagueness Spread

● Vagueness Intensity

Problem Definition & Proposal


● The ratio of the number of ontological elements (classes, relations and datatypes) that are vague, divided by the total number of elements.

● This metric reflects the extent to which vagueness is present in the ontology.

● It also provides an indication of the ontology's potential comprehensibility and shareability:

● An ontology with a high value of vagueness spread is less explicit and shareable than an ontology with a low value.

● Calculation of this metric can be done semi-automatically using the vagueness detector we have described.

Vagueness Spread


● The degree to which the ontology's users disagree on the validity of the (potential) instances of the ontology elements.

● The higher this disagreement is for an element, the more problems the element is likely to cause.

● Metric Calculation:● Consider a sample set of vague element instances, ● Have potential ontology users denote whether and to what extent

they believe these instances are valid● Measure the inter-agreement between users (e.g. by using Cohen’s

kappa)

Vagueness Intensity

Q3: Vagueness Explicitness

● Can we make the intended meaning of vague elements in ontologies more explicit than it typically is?

● For example, ontologies typically do not explicitly state which of their elements are vague and which are not.

● Why is this a problem?

● “Fat Person” as “A person weighing many kilos” is vague!● “Fat Person” as “A person with a BMI greater than 25” is not!

● What are the odds that a typical ontology user will immediately see this distinction without prior notice?

Problem Definition

Q3: Vagueness ExplicitnessOur Proposal: Vagueness Aware Ontologies

Vagueness-aware ontologies are “ontologies

whose vague entities are accompanied by

meta-information that describes the nature

and characteristics of their vagueness”

Q3: Vagueness ExplicitnessWhat should be made explicit?

E.g. “Low Budget” has quantitative vagueness while “Expert Consultant”

qualitative.

“E.g. “Strategic Client" is vague in the dimension of the generated revenue

”

“E.g. “Strategic Client" is vague in the dimension of the generated revenue in the context of Financial Reporting”

E.g. “Strategic Client" is vague in the dimension of the generated revenue according to the Financial Manager.

Vagueness Type

Vagueness dimensions

Applicability Context

Entity Creator

E.g. “Tall Person” is vague.Vagueness Existence

Q3: Vagueness ExplicitnessThe Vagueness Ontology

Vagueness Ontology

● Q1: What entities have been explicitly defined either as vague or non-vague?

● Q2: What entities are vague, in what contexts and according to whom?

● Q3: What entities have been defined both as vague and non-vague at the same time and why?

● Q4: What entities of a specific type (e.g., classes) have been defined either as vague or non-vague?

● Q5: What entities are characterised by a specific vagueness type?

● Q6: What entities have quantitative vagueness, in what dimensions and what is the context of their dimensions (if any)?

Supported Competency Questions

Vagueness Ontology

● The object property isExpertInResearchArea is considered vague by John Doe in the context of researcher hiring.

● Moreover, it is quantitatively vague since, for him, expertise is judged by the number of publications and projects.

● These two different dimensions he thinks are applicable in different contexts:

● Number of relevant publications in Academia

● Number of relevant projects in Industry.

Example Scenario

Vagueness OntologyExample Scenario

Vagueness Ontology

● Vagueness Ontology is meant to be used by both producers and consumers of ontologies and semantic data.

● The former to annotate the vague part of their produced ontologies with relevant vagueness metainformation

● The latter to query this metainformation and use it to make a better use of the data.

● The annotation task should ideally take place in the course of the ontology's construction and evolution process.

Intended Usage

Vagueness OntologyUsage and Benefits

Vagueness Ontology Usage Expected Benefits

Structuring Data with a Vague Ontology

• Communicate the meaning of the vague

elements to the domain experts.

• Use the metamodel to characterize the created

data's vagueness.

• Make the job of the experts

easier and faster and

reduce disagreements

among them.

Utilizing Vague Semantic Data in an Ontology-Based System

• Check which data is vague.

• Use the properties of the vague elements to

provide vagueness-related explanations to the

users.

• Know a priori which data

may affect the system's

effectiveness.

Integrating Vague Semantic Datasets

• Compare same vague elements across datasets

according to their vagueness type and

dimensions

• Know a priori which data

may affect the system's

effectiveness.

Evaluating Vague Semantic Datasets for Reuse

• Query the metamodel to check the vagueness

compatibility of the dataset with the intended

domain and application scenario.

• Avoid re-using (parts of)

datasets that are not

compatible to own

interpretation of vagueness.

Vagueness OntologyDocumentation and Resources

Documentation: http://www.essepuntato.it/2013/10/vagueness/documentation.html Examples: http://www.essepuntato.it/2013/10/vagueness/examples.html

Ongoing and Future Work

● Develop ontology authoring tool that:

1. Takes as input an ontology.

2. Detects automatically vague entities.

3. Guides the user into annotating them with the Vagueness Ontology in a Q&A manner.

4. Gives as output a vagueness annotation for the given ontology.

Creation of Vagueness-Aware Ontologies

Ongoing and Future Work

● Refine/expand/evaluate the current vagueness metrics we have defined.

● Devise methods and tools for their effective and efficient calculation.

Vagueness-Based Evaluation of Ontologies

● Identify vagueness-related reasoning tasks that are useful when working with vague ontologies.

● Expand/revise the Vagueness Ontology accordingly.

Reasoning with Vagueness-Aware Ontologies

Thank you for your attention!

Dr. Panos AlexopoulosSemantic Applications Research Manager

Email: [email protected]

Web: www.panosalexopoulos.com

LinkedIn: www.linkedin.com/in/panosalexopoulos

Twitter: @PAlexop

Detecting, Measuring and Representing Vagueness in Ontologies

Technology

Transcript of Detecting, Measuring and Representing Vagueness in Ontologies