GENI Instrumentation and Measurement System -

Schema

Martin Swany

Schema

• Generally: model of objects and their relationships

• We should talk in terms of the general “schema” in this sense

• Renderings later– XML schema– SQL schema– IPFIX– JSON

I&M Schema Requirements• Consistent basic representation of

measurement and instrumentation data inside and outside of a slice

• Flexibility of encoding and transport– timestamp and value or values

• Expressive metadata• Easily extensible• Reusable components and models• Reasonable relationship to RSpec

perfSONAR Model

• perfSONAR is an internationally-adopted framework for instrumentation and measurement– Based on schemata and protocols defined in

the *Grid Forum over the last 10 years– Recognized by the NSF as a key technology;

recent NSF-funded workshop

• Again, consider the model in the abstract rather than the XML encoding or RNC files

perfSONAR Schema• Key Goals: Extensibility, Normalization,

Readability• Break representation of performance

measurements down into basic elements• Data and Metadata• Measurement Data

– A set of of measurement events that have some value or values at a particular time

• Measurement Metadata– The details about the set of measurement data

Schema Normalization

• Can simplify the database representation for many types of measurement data– While optimizations are possible, many

measurement types can be viewed as one value measured over time

• Assists Combination/Concatenation of metrics– Creating derived metrics

• Normalization helps with inferring relationships between types of metrics

Schema Basic Elements - Metadata

• Subject (Noun)– The measured/tested entity

• EventType (Verb)– What type of measurement or event

occurred, or instrumented parameter was read

– Characteristic, tool output, or generic event

• Parameters (Adjectives and Adverbs)– How, or under what conditions, did this

event occur?

Schema Basic Elements - Data

• Some sort of value - Datum– Existence of an event might point to the

case where there no additional value• As in “Link up/down” or threshold events

• Time– Is extensible since various representations

are appropriate in different cases• E.g. UNIX timestamp vs NTP time

A MessageMessageMessage

Metadata

Data

An Object Store

Store

Metadata

Data

A Data is Linked to a Metadata

Metadata

<id>someId</id>

Data

<metadataIdRef> someId</metadataIdRef>

A Metadata may be linked to another

Metadata

<id>someId</id>

Metadata<id>someOtherId</id>

<metadataIdRef> someId</metadataIdRef>

Schema Namespaces

• Observation: all measurements have some sort of Data and Time

• All measurements can be described by the Metadata identifying who, what and how

• The specific structures of the Data and Metadata elements depend on the measurement

• Approach: Consistently use Data and Metadata elements and vary the namespaces of the specific elements

Schema Namespaces - 2

• We encode the measurement/event type in the namespace– And as a standalone element

• Some components of the system can pass Data and Metadata elements through without understanding their specific structure

• Allows and implementation to decide whether it supports a particular type of data or not

• Allows validation based on extended (namespace-specific) schemata

Schema Namespaces and Extensibility• One key to extensibility is the use of

hierarchy with delegation– Similar to OIDs in the IETF management

world

• The OGF NM-WG defined a hierarchy of network characteristics– Good starting point

• However, not all tools are cleanly mapped onto the Characteristic space– Often a matter of some debate

Schema Namespaces and Extensibility

• Organization-rooted tools namespace addresses this

• Some top-level tools• ping, traceroute

• Easy to add new tools in organization-specific namespaces

• Performance Event Repository– Add a schema and get a URI– Add Java classes

Linking Metadata

• Metadata can be linked in two ways– Merge chaining allows for elements to be

reused and a complete metadata can be built– Operation chaining requests or describes

operations on data sets• Representation of data provenance

AA

BB

ABAB

AA

BB

B(A)

Resources, Subjects, Topology

• perfSONAR has a topology schema called UNIS – Unified Network Information Schema

• Related to the control frameworks’ Resource Specification

• Measurement and Instrumentation must be related to the resources themselves

end