OCCI Monitoring at OGF42 - Concepts and demo

176
OCCI monitoring extension Augusto Ciuffoletti OCCI monitoring extension and its proof of concept Augusto Ciuffoletti Dept. of Computer Science - Univ. of Pisa September 5, 2014

Transcript of OCCI Monitoring at OGF42 - Concepts and demo

Page 1: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

OCCI monitoring extensionand its proof of concept

Augusto Ciuffoletti

Dept. of Computer Science - Univ. of Pisa

September 5, 2014

Page 2: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The OCCI framework

I There is an interface between the user of a cloud serviceand the cloud service itself

I Data entities that describe the service traverse thisinterface during its provisioning

I The protocol used during this conversation follows theREST paradigm:

I the user plays the role of the clientI the conversation follows the HTTP protocolI responses are cacheable, as far as possible

I OCCI proposes a minimalistic conceptual framework (orontology) for the entities used to describe the service

Page 3: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The OCCI framework

I There is an interface between the user of a cloud serviceand the cloud service itself

I Data entities that describe the service traverse thisinterface during its provisioning

I The protocol used during this conversation follows theREST paradigm:

I the user plays the role of the clientI the conversation follows the HTTP protocolI responses are cacheable, as far as possible

I OCCI proposes a minimalistic conceptual framework (orontology) for the entities used to describe the service

Page 4: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The OCCI framework

I There is an interface between the user of a cloud serviceand the cloud service itself

I Data entities that describe the service traverse thisinterface during its provisioning

I The protocol used during this conversation follows theREST paradigm:

I the user plays the role of the clientI the conversation follows the HTTP protocolI responses are cacheable, as far as possible

I OCCI proposes a minimalistic conceptual framework (orontology) for the entities used to describe the service

Page 5: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The OCCI framework

I There is an interface between the user of a cloud serviceand the cloud service itself

I Data entities that describe the service traverse thisinterface during its provisioning

I The protocol used during this conversation follows theREST paradigm:

I the user plays the role of the clientI the conversation follows the HTTP protocolI responses are cacheable, as far as possible

I OCCI proposes a minimalistic conceptual framework (orontology) for the entities used to describe the service

Page 6: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The OCCI framework

I There is an interface between the user of a cloud serviceand the cloud service itself

I Data entities that describe the service traverse thisinterface during its provisioning

I The protocol used during this conversation follows theREST paradigm:

I the user plays the role of the clientI the conversation follows the HTTP protocolI responses are cacheable, as far as possible

I OCCI proposes a minimalistic conceptual framework (orontology) for the entities used to describe the service

Page 7: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The OCCI framework

I There is an interface between the user of a cloud serviceand the cloud service itself

I Data entities that describe the service traverse thisinterface during its provisioning

I The protocol used during this conversation follows theREST paradigm:

I the user plays the role of the clientI the conversation follows the HTTP protocolI responses are cacheable, as far as possible

I OCCI proposes a minimalistic conceptual framework (orontology) for the entities used to describe the service

Page 8: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The OCCI framework

I There is an interface between the user of a cloud serviceand the cloud service itself

I Data entities that describe the service traverse thisinterface during its provisioning

I The protocol used during this conversation follows theREST paradigm:

I the user plays the role of the clientI the conversation follows the HTTP protocolI responses are cacheable, as far as possible

I OCCI proposes a minimalistic conceptual framework (orontology) for the entities used to describe the service

Page 9: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The OCCI core concepts

I Anything is an entity, and is identified with an URII A relationship between entities is an entity

I we distinguish resource entities and link entities(relationship)

I There are many kinds of entities, with distinguishingattributes

I An entity of a certain kind can be integrated withmixins that carry more attributes or bind existing ones

Page 10: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The OCCI core concepts

I Anything is an entity, and is identified with an URII A relationship between entities is an entity

I we distinguish resource entities and link entities(relationship)

I There are many kinds of entities, with distinguishingattributes

I An entity of a certain kind can be integrated withmixins that carry more attributes or bind existing ones

Page 11: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The OCCI core concepts

I Anything is an entity, and is identified with an URII A relationship between entities is an entity

I we distinguish resource entities and link entities(relationship)

I There are many kinds of entities, with distinguishingattributes

I An entity of a certain kind can be integrated withmixins that carry more attributes or bind existing ones

Page 12: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The OCCI core concepts

I Anything is an entity, and is identified with an URII A relationship between entities is an entity

I we distinguish resource entities and link entities(relationship)

I There are many kinds of entities, with distinguishingattributes

I An entity of a certain kind can be integrated withmixins that carry more attributes or bind existing ones

Page 13: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The OCCI core concepts

I Anything is an entity, and is identified with an URII A relationship between entities is an entity

I we distinguish resource entities and link entities(relationship)

I There are many kinds of entities, with distinguishingattributes

I An entity of a certain kind can be integrated withmixins that carry more attributes or bind existing ones

Page 14: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

OCCI IaaS interface

I The first use case for OCCI, adopted as an openstandard

I Resources are processors, storage, networks.I Links are network interfaces, and processor/storage

associationsI Mixins add OpSys attributes to a processor, a

filesystem to a storage, etc.

I However OCCI is designed to smoothly adapt to diverseuse cases

I Here we want to propose its application to monitoringthe performance of cloud resources

Page 15: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

OCCI IaaS interface

I The first use case for OCCI, adopted as an openstandard

I Resources are processors, storage, networks.I Links are network interfaces, and processor/storage

associationsI Mixins add OpSys attributes to a processor, a

filesystem to a storage, etc.

I However OCCI is designed to smoothly adapt to diverseuse cases

I Here we want to propose its application to monitoringthe performance of cloud resources

Page 16: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

OCCI IaaS interface

I The first use case for OCCI, adopted as an openstandard

I Resources are processors, storage, networks.I Links are network interfaces, and processor/storage

associationsI Mixins add OpSys attributes to a processor, a

filesystem to a storage, etc.

I However OCCI is designed to smoothly adapt to diverseuse cases

I Here we want to propose its application to monitoringthe performance of cloud resources

Page 17: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

OCCI IaaS interface

I The first use case for OCCI, adopted as an openstandard

I Resources are processors, storage, networks.I Links are network interfaces, and processor/storage

associationsI Mixins add OpSys attributes to a processor, a

filesystem to a storage, etc.

I However OCCI is designed to smoothly adapt to diverseuse cases

I Here we want to propose its application to monitoringthe performance of cloud resources

Page 18: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

OCCI IaaS interface

I The first use case for OCCI, adopted as an openstandard

I Resources are processors, storage, networks.I Links are network interfaces, and processor/storage

associationsI Mixins add OpSys attributes to a processor, a

filesystem to a storage, etc.

I However OCCI is designed to smoothly adapt to diverseuse cases

I Here we want to propose its application to monitoringthe performance of cloud resources

Page 19: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

OCCI IaaS interface

I The first use case for OCCI, adopted as an openstandard

I Resources are processors, storage, networks.I Links are network interfaces, and processor/storage

associationsI Mixins add OpSys attributes to a processor, a

filesystem to a storage, etc.

I However OCCI is designed to smoothly adapt to diverseuse cases

I Here we want to propose its application to monitoringthe performance of cloud resources

Page 20: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Motivation

I The monitoring of resource performance is a key issueto implement:

I Service Level Agreement, a defined target to obtain userconfidence

I fault-tolerance targets defined by the user

I The user wants to customize resource monitoringI the user may be in his turn a service providerI the user may simply want to verify the quality of the

serviceI major cloud providers providers offer resource

monitoring as a part of the service

I Standardization matters!I Consider the case of a composite service (many

providers)

I We propose a simple ontology for resource monitoringthat is aligned with OCCI

Page 21: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Motivation

I The monitoring of resource performance is a key issueto implement:

I Service Level Agreement, a defined target to obtain userconfidence

I fault-tolerance targets defined by the user

I The user wants to customize resource monitoringI the user may be in his turn a service providerI the user may simply want to verify the quality of the

serviceI major cloud providers providers offer resource

monitoring as a part of the service

I Standardization matters!I Consider the case of a composite service (many

providers)

I We propose a simple ontology for resource monitoringthat is aligned with OCCI

Page 22: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Motivation

I The monitoring of resource performance is a key issueto implement:

I Service Level Agreement, a defined target to obtain userconfidence

I fault-tolerance targets defined by the user

I The user wants to customize resource monitoringI the user may be in his turn a service providerI the user may simply want to verify the quality of the

serviceI major cloud providers providers offer resource

monitoring as a part of the service

I Standardization matters!I Consider the case of a composite service (many

providers)

I We propose a simple ontology for resource monitoringthat is aligned with OCCI

Page 23: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Motivation

I The monitoring of resource performance is a key issueto implement:

I Service Level Agreement, a defined target to obtain userconfidence

I fault-tolerance targets defined by the user

I The user wants to customize resource monitoringI the user may be in his turn a service providerI the user may simply want to verify the quality of the

serviceI major cloud providers providers offer resource

monitoring as a part of the service

I Standardization matters!I Consider the case of a composite service (many

providers)

I We propose a simple ontology for resource monitoringthat is aligned with OCCI

Page 24: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Motivation

I The monitoring of resource performance is a key issueto implement:

I Service Level Agreement, a defined target to obtain userconfidence

I fault-tolerance targets defined by the user

I The user wants to customize resource monitoringI the user may be in his turn a service providerI the user may simply want to verify the quality of the

serviceI major cloud providers providers offer resource

monitoring as a part of the service

I Standardization matters!I Consider the case of a composite service (many

providers)

I We propose a simple ontology for resource monitoringthat is aligned with OCCI

Page 25: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Motivation

I The monitoring of resource performance is a key issueto implement:

I Service Level Agreement, a defined target to obtain userconfidence

I fault-tolerance targets defined by the user

I The user wants to customize resource monitoringI the user may be in his turn a service providerI the user may simply want to verify the quality of the

serviceI major cloud providers providers offer resource

monitoring as a part of the service

I Standardization matters!I Consider the case of a composite service (many

providers)

I We propose a simple ontology for resource monitoringthat is aligned with OCCI

Page 26: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Motivation

I The monitoring of resource performance is a key issueto implement:

I Service Level Agreement, a defined target to obtain userconfidence

I fault-tolerance targets defined by the user

I The user wants to customize resource monitoringI the user may be in his turn a service providerI the user may simply want to verify the quality of the

serviceI major cloud providers providers offer resource

monitoring as a part of the service

I Standardization matters!I Consider the case of a composite service (many

providers)

I We propose a simple ontology for resource monitoringthat is aligned with OCCI

Page 27: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Motivation

I The monitoring of resource performance is a key issueto implement:

I Service Level Agreement, a defined target to obtain userconfidence

I fault-tolerance targets defined by the user

I The user wants to customize resource monitoringI the user may be in his turn a service providerI the user may simply want to verify the quality of the

serviceI major cloud providers providers offer resource

monitoring as a part of the service

I Standardization matters!I Consider the case of a composite service (many

providers)

I We propose a simple ontology for resource monitoringthat is aligned with OCCI

Page 28: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Motivation

I The monitoring of resource performance is a key issueto implement:

I Service Level Agreement, a defined target to obtain userconfidence

I fault-tolerance targets defined by the user

I The user wants to customize resource monitoringI the user may be in his turn a service providerI the user may simply want to verify the quality of the

serviceI major cloud providers providers offer resource

monitoring as a part of the service

I Standardization matters!I Consider the case of a composite service (many

providers)

I We propose a simple ontology for resource monitoringthat is aligned with OCCI

Page 29: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Motivation

I The monitoring of resource performance is a key issueto implement:

I Service Level Agreement, a defined target to obtain userconfidence

I fault-tolerance targets defined by the user

I The user wants to customize resource monitoringI the user may be in his turn a service providerI the user may simply want to verify the quality of the

serviceI major cloud providers providers offer resource

monitoring as a part of the service

I Standardization matters!I Consider the case of a composite service (many

providers)

I We propose a simple ontology for resource monitoringthat is aligned with OCCI

Page 30: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A monitoring framework

I Monitoring is made of three basic activities:I extract performance parameters from the target

ResourceI aggregate them and compute the metric of interestI deliver the measurement to the relevant party

I The last two steps consist of aggregation and renderingof data

I this makes a candidate for a new Resource kind

I The first step entails the collaboration among entitiesI this makes a candidate for a new Link kind

I The resource kind is named Sensor, and the link kindCollector

I and this is bare minimum for a stand-alone monitoringframework

I The OCCI monitoring framework complements anyOCCI framework

I it handles any type of Resource.

Page 31: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A monitoring framework

I Monitoring is made of three basic activities:I extract performance parameters from the target

ResourceI aggregate them and compute the metric of interestI deliver the measurement to the relevant party

I The last two steps consist of aggregation and renderingof data

I this makes a candidate for a new Resource kind

I The first step entails the collaboration among entitiesI this makes a candidate for a new Link kind

I The resource kind is named Sensor, and the link kindCollector

I and this is bare minimum for a stand-alone monitoringframework

I The OCCI monitoring framework complements anyOCCI framework

I it handles any type of Resource.

Page 32: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A monitoring framework

I Monitoring is made of three basic activities:I extract performance parameters from the target

ResourceI aggregate them and compute the metric of interestI deliver the measurement to the relevant party

I The last two steps consist of aggregation and renderingof data

I this makes a candidate for a new Resource kind

I The first step entails the collaboration among entitiesI this makes a candidate for a new Link kind

I The resource kind is named Sensor, and the link kindCollector

I and this is bare minimum for a stand-alone monitoringframework

I The OCCI monitoring framework complements anyOCCI framework

I it handles any type of Resource.

Page 33: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A monitoring framework

I Monitoring is made of three basic activities:I extract performance parameters from the target

ResourceI aggregate them and compute the metric of interestI deliver the measurement to the relevant party

I The last two steps consist of aggregation and renderingof data

I this makes a candidate for a new Resource kind

I The first step entails the collaboration among entitiesI this makes a candidate for a new Link kind

I The resource kind is named Sensor, and the link kindCollector

I and this is bare minimum for a stand-alone monitoringframework

I The OCCI monitoring framework complements anyOCCI framework

I it handles any type of Resource.

Page 34: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A monitoring framework

I Monitoring is made of three basic activities:I extract performance parameters from the target

ResourceI aggregate them and compute the metric of interestI deliver the measurement to the relevant party

I The last two steps consist of aggregation and renderingof data

I this makes a candidate for a new Resource kind

I The first step entails the collaboration among entitiesI this makes a candidate for a new Link kind

I The resource kind is named Sensor, and the link kindCollector

I and this is bare minimum for a stand-alone monitoringframework

I The OCCI monitoring framework complements anyOCCI framework

I it handles any type of Resource.

Page 35: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A monitoring framework

I Monitoring is made of three basic activities:I extract performance parameters from the target

ResourceI aggregate them and compute the metric of interestI deliver the measurement to the relevant party

I The last two steps consist of aggregation and renderingof data

I this makes a candidate for a new Resource kind

I The first step entails the collaboration among entitiesI this makes a candidate for a new Link kind

I The resource kind is named Sensor, and the link kindCollector

I and this is bare minimum for a stand-alone monitoringframework

I The OCCI monitoring framework complements anyOCCI framework

I it handles any type of Resource.

Page 36: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A monitoring framework

I Monitoring is made of three basic activities:I extract performance parameters from the target

ResourceI aggregate them and compute the metric of interestI deliver the measurement to the relevant party

I The last two steps consist of aggregation and renderingof data

I this makes a candidate for a new Resource kind

I The first step entails the collaboration among entitiesI this makes a candidate for a new Link kind

I The resource kind is named Sensor, and the link kindCollector

I and this is bare minimum for a stand-alone monitoringframework

I The OCCI monitoring framework complements anyOCCI framework

I it handles any type of Resource.

Page 37: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A monitoring framework

I Monitoring is made of three basic activities:I extract performance parameters from the target

ResourceI aggregate them and compute the metric of interestI deliver the measurement to the relevant party

I The last two steps consist of aggregation and renderingof data

I this makes a candidate for a new Resource kind

I The first step entails the collaboration among entitiesI this makes a candidate for a new Link kind

I The resource kind is named Sensor, and the link kindCollector

I and this is bare minimum for a stand-alone monitoringframework

I The OCCI monitoring framework complements anyOCCI framework

I it handles any type of Resource.

Page 38: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A monitoring framework

I Monitoring is made of three basic activities:I extract performance parameters from the target

ResourceI aggregate them and compute the metric of interestI deliver the measurement to the relevant party

I The last two steps consist of aggregation and renderingof data

I this makes a candidate for a new Resource kind

I The first step entails the collaboration among entitiesI this makes a candidate for a new Link kind

I The resource kind is named Sensor, and the link kindCollector

I and this is bare minimum for a stand-alone monitoringframework

I The OCCI monitoring framework complements anyOCCI framework

I it handles any type of Resource.

Page 39: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A monitoring framework

I Monitoring is made of three basic activities:I extract performance parameters from the target

ResourceI aggregate them and compute the metric of interestI deliver the measurement to the relevant party

I The last two steps consist of aggregation and renderingof data

I this makes a candidate for a new Resource kind

I The first step entails the collaboration among entitiesI this makes a candidate for a new Link kind

I The resource kind is named Sensor, and the link kindCollector

I and this is bare minimum for a stand-alone monitoringframework

I The OCCI monitoring framework complements anyOCCI framework

I it handles any type of Resource.

Page 40: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A monitoring framework

I Monitoring is made of three basic activities:I extract performance parameters from the target

ResourceI aggregate them and compute the metric of interestI deliver the measurement to the relevant party

I The last two steps consist of aggregation and renderingof data

I this makes a candidate for a new Resource kind

I The first step entails the collaboration among entitiesI this makes a candidate for a new Link kind

I The resource kind is named Sensor, and the link kindCollector

I and this is bare minimum for a stand-alone monitoringframework

I The OCCI monitoring framework complements anyOCCI framework

I it handles any type of Resource.

Page 41: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A monitoring framework

I Monitoring is made of three basic activities:I extract performance parameters from the target

ResourceI aggregate them and compute the metric of interestI deliver the measurement to the relevant party

I The last two steps consist of aggregation and renderingof data

I this makes a candidate for a new Resource kind

I The first step entails the collaboration among entitiesI this makes a candidate for a new Link kind

I The resource kind is named Sensor, and the link kindCollector

I and this is bare minimum for a stand-alone monitoringframework

I The OCCI monitoring framework complements anyOCCI framework

I it handles any type of Resource.

Page 42: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A Sensor

I It is a distiguished activity that needs the provision ofcloud resources

I Tightly integrated in cloud infrastructureI Under control of the providerI Tuned using user requests

I The provider defines the available aggregation andpublishing capabilities

I The user instantiates the Sensor as a composition ofsuch capabilities

Page 43: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A Sensor

I It is a distiguished activity that needs the provision ofcloud resources

I Tightly integrated in cloud infrastructureI Under control of the providerI Tuned using user requests

I The provider defines the available aggregation andpublishing capabilities

I The user instantiates the Sensor as a composition ofsuch capabilities

Page 44: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A Sensor

I It is a distiguished activity that needs the provision ofcloud resources

I Tightly integrated in cloud infrastructureI Under control of the providerI Tuned using user requests

I The provider defines the available aggregation andpublishing capabilities

I The user instantiates the Sensor as a composition ofsuch capabilities

Page 45: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A Sensor

I It is a distiguished activity that needs the provision ofcloud resources

I Tightly integrated in cloud infrastructureI Under control of the providerI Tuned using user requests

I The provider defines the available aggregation andpublishing capabilities

I The user instantiates the Sensor as a composition ofsuch capabilities

Page 46: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A Sensor

I It is a distiguished activity that needs the provision ofcloud resources

I Tightly integrated in cloud infrastructureI Under control of the providerI Tuned using user requests

I The provider defines the available aggregation andpublishing capabilities

I The user instantiates the Sensor as a composition ofsuch capabilities

Page 47: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A Sensor

I It is a distiguished activity that needs the provision ofcloud resources

I Tightly integrated in cloud infrastructureI Under control of the providerI Tuned using user requests

I The provider defines the available aggregation andpublishing capabilities

I The user instantiates the Sensor as a composition ofsuch capabilities

Page 48: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Describing a Sensor

I Any Sensor has a few generic featuresI ...they can be included in a standard definition of a

SensorI When the sensor operatesI How frequently the sensor produces a new measurementI They are timing attributes

I Other features are specific for the providerI ...they are defined as mixins for the sensorI How data are aggregated (low pass, patterns etc.)I How data are published (archive, email, streaming etc.)

I There is no limit to the semantic of the mixinsI however the hooks to connect a Sensor to a Collector

must be defined

Page 49: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Describing a Sensor

I Any Sensor has a few generic featuresI ...they can be included in a standard definition of a

SensorI When the sensor operatesI How frequently the sensor produces a new measurementI They are timing attributes

I Other features are specific for the providerI ...they are defined as mixins for the sensorI How data are aggregated (low pass, patterns etc.)I How data are published (archive, email, streaming etc.)

I There is no limit to the semantic of the mixinsI however the hooks to connect a Sensor to a Collector

must be defined

Page 50: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Describing a Sensor

I Any Sensor has a few generic featuresI ...they can be included in a standard definition of a

SensorI When the sensor operatesI How frequently the sensor produces a new measurementI They are timing attributes

I Other features are specific for the providerI ...they are defined as mixins for the sensorI How data are aggregated (low pass, patterns etc.)I How data are published (archive, email, streaming etc.)

I There is no limit to the semantic of the mixinsI however the hooks to connect a Sensor to a Collector

must be defined

Page 51: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Describing a Sensor

I Any Sensor has a few generic featuresI ...they can be included in a standard definition of a

SensorI When the sensor operatesI How frequently the sensor produces a new measurementI They are timing attributes

I Other features are specific for the providerI ...they are defined as mixins for the sensorI How data are aggregated (low pass, patterns etc.)I How data are published (archive, email, streaming etc.)

I There is no limit to the semantic of the mixinsI however the hooks to connect a Sensor to a Collector

must be defined

Page 52: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Describing a Sensor

I Any Sensor has a few generic featuresI ...they can be included in a standard definition of a

SensorI When the sensor operatesI How frequently the sensor produces a new measurementI They are timing attributes

I Other features are specific for the providerI ...they are defined as mixins for the sensorI How data are aggregated (low pass, patterns etc.)I How data are published (archive, email, streaming etc.)

I There is no limit to the semantic of the mixinsI however the hooks to connect a Sensor to a Collector

must be defined

Page 53: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Describing a Sensor

I Any Sensor has a few generic featuresI ...they can be included in a standard definition of a

SensorI When the sensor operatesI How frequently the sensor produces a new measurementI They are timing attributes

I Other features are specific for the providerI ...they are defined as mixins for the sensorI How data are aggregated (low pass, patterns etc.)I How data are published (archive, email, streaming etc.)

I There is no limit to the semantic of the mixinsI however the hooks to connect a Sensor to a Collector

must be defined

Page 54: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Describing a Sensor

I Any Sensor has a few generic featuresI ...they can be included in a standard definition of a

SensorI When the sensor operatesI How frequently the sensor produces a new measurementI They are timing attributes

I Other features are specific for the providerI ...they are defined as mixins for the sensorI How data are aggregated (low pass, patterns etc.)I How data are published (archive, email, streaming etc.)

I There is no limit to the semantic of the mixinsI however the hooks to connect a Sensor to a Collector

must be defined

Page 55: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Describing a Sensor

I Any Sensor has a few generic featuresI ...they can be included in a standard definition of a

SensorI When the sensor operatesI How frequently the sensor produces a new measurementI They are timing attributes

I Other features are specific for the providerI ...they are defined as mixins for the sensorI How data are aggregated (low pass, patterns etc.)I How data are published (archive, email, streaming etc.)

I There is no limit to the semantic of the mixinsI however the hooks to connect a Sensor to a Collector

must be defined

Page 56: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Describing a Sensor

I Any Sensor has a few generic featuresI ...they can be included in a standard definition of a

SensorI When the sensor operatesI How frequently the sensor produces a new measurementI They are timing attributes

I Other features are specific for the providerI ...they are defined as mixins for the sensorI How data are aggregated (low pass, patterns etc.)I How data are published (archive, email, streaming etc.)

I There is no limit to the semantic of the mixinsI however the hooks to connect a Sensor to a Collector

must be defined

Page 57: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Describing a Sensor

I Any Sensor has a few generic featuresI ...they can be included in a standard definition of a

SensorI When the sensor operatesI How frequently the sensor produces a new measurementI They are timing attributes

I Other features are specific for the providerI ...they are defined as mixins for the sensorI How data are aggregated (low pass, patterns etc.)I How data are published (archive, email, streaming etc.)

I There is no limit to the semantic of the mixinsI however the hooks to connect a Sensor to a Collector

must be defined

Page 58: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Describing a Sensor

I Any Sensor has a few generic featuresI ...they can be included in a standard definition of a

SensorI When the sensor operatesI How frequently the sensor produces a new measurementI They are timing attributes

I Other features are specific for the providerI ...they are defined as mixins for the sensorI How data are aggregated (low pass, patterns etc.)I How data are published (archive, email, streaming etc.)

I There is no limit to the semantic of the mixinsI however the hooks to connect a Sensor to a Collector

must be defined

Page 59: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A Collector

I Represents a flow of measurements between a OCCIResource and a Sensor

I ... yes, the source can be a Sensor in its turn

I The provider has control on the available measurements

I The user has control on the selection and theconfiguration of the Collectors

I Cross provider measurements can be implementedI ... to accomodate the utilization of several providers

with a unique dashboard Sensor

Page 60: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A Collector

I Represents a flow of measurements between a OCCIResource and a Sensor

I ... yes, the source can be a Sensor in its turn

I The provider has control on the available measurements

I The user has control on the selection and theconfiguration of the Collectors

I Cross provider measurements can be implementedI ... to accomodate the utilization of several providers

with a unique dashboard Sensor

Page 61: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A Collector

I Represents a flow of measurements between a OCCIResource and a Sensor

I ... yes, the source can be a Sensor in its turn

I The provider has control on the available measurements

I The user has control on the selection and theconfiguration of the Collectors

I Cross provider measurements can be implementedI ... to accomodate the utilization of several providers

with a unique dashboard Sensor

Page 62: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A Collector

I Represents a flow of measurements between a OCCIResource and a Sensor

I ... yes, the source can be a Sensor in its turn

I The provider has control on the available measurements

I The user has control on the selection and theconfiguration of the Collectors

I Cross provider measurements can be implementedI ... to accomodate the utilization of several providers

with a unique dashboard Sensor

Page 63: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A Collector

I Represents a flow of measurements between a OCCIResource and a Sensor

I ... yes, the source can be a Sensor in its turn

I The provider has control on the available measurements

I The user has control on the selection and theconfiguration of the Collectors

I Cross provider measurements can be implementedI ... to accomodate the utilization of several providers

with a unique dashboard Sensor

Page 64: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A Collector

I Represents a flow of measurements between a OCCIResource and a Sensor

I ... yes, the source can be a Sensor in its turn

I The provider has control on the available measurements

I The user has control on the selection and theconfiguration of the Collectors

I Cross provider measurements can be implementedI ... to accomodate the utilization of several providers

with a unique dashboard Sensor

Page 65: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Describing a Collector

I As in the case of the Sensor there are generic attributesof a collector:

I The sampling periodI The accuracy of the sampling periodI ... again, just timing

I Other attributes are defined by provider-specific mixinswith an arbitrary semantic

I ...the metric that is measured (throughput, free space,temperature etc.)

Page 66: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Describing a Collector

I As in the case of the Sensor there are generic attributesof a collector:

I The sampling periodI The accuracy of the sampling periodI ... again, just timing

I Other attributes are defined by provider-specific mixinswith an arbitrary semantic

I ...the metric that is measured (throughput, free space,temperature etc.)

Page 67: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Describing a Collector

I As in the case of the Sensor there are generic attributesof a collector:

I The sampling periodI The accuracy of the sampling periodI ... again, just timing

I Other attributes are defined by provider-specific mixinswith an arbitrary semantic

I ...the metric that is measured (throughput, free space,temperature etc.)

Page 68: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Describing a Collector

I As in the case of the Sensor there are generic attributesof a collector:

I The sampling periodI The accuracy of the sampling periodI ... again, just timing

I Other attributes are defined by provider-specific mixinswith an arbitrary semantic

I ...the metric that is measured (throughput, free space,temperature etc.)

Page 69: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Describing a Collector

I As in the case of the Sensor there are generic attributesof a collector:

I The sampling periodI The accuracy of the sampling periodI ... again, just timing

I Other attributes are defined by provider-specific mixinswith an arbitrary semantic

I ...the metric that is measured (throughput, free space,temperature etc.)

Page 70: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Describing a Collector

I As in the case of the Sensor there are generic attributesof a collector:

I The sampling periodI The accuracy of the sampling periodI ... again, just timing

I Other attributes are defined by provider-specific mixinswith an arbitrary semantic

I ...the metric that is measured (throughput, free space,temperature etc.)

Page 71: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A monitoring-aware Resource

I A Resource that is the target of a monitoring activitymay be explicitely characterized with a CollectorEnd-Point mixin

I Such a mixin contains the description of the monitoringmodality for that resource

I for instance, the location of a log file

I The presence of this mixin allows cross-providerinteroperability

I As an alternative, the modality can be left implicit(default)

Page 72: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A monitoring-aware Resource

I A Resource that is the target of a monitoring activitymay be explicitely characterized with a CollectorEnd-Point mixin

I Such a mixin contains the description of the monitoringmodality for that resource

I for instance, the location of a log file

I The presence of this mixin allows cross-providerinteroperability

I As an alternative, the modality can be left implicit(default)

Page 73: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A monitoring-aware Resource

I A Resource that is the target of a monitoring activitymay be explicitely characterized with a CollectorEnd-Point mixin

I Such a mixin contains the description of the monitoringmodality for that resource

I for instance, the location of a log file

I The presence of this mixin allows cross-providerinteroperability

I As an alternative, the modality can be left implicit(default)

Page 74: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A monitoring-aware Resource

I A Resource that is the target of a monitoring activitymay be explicitely characterized with a CollectorEnd-Point mixin

I Such a mixin contains the description of the monitoringmodality for that resource

I for instance, the location of a log file

I The presence of this mixin allows cross-providerinteroperability

I As an alternative, the modality can be left implicit(default)

Page 75: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A monitoring-aware Resource

I A Resource that is the target of a monitoring activitymay be explicitely characterized with a CollectorEnd-Point mixin

I Such a mixin contains the description of the monitoringmodality for that resource

I for instance, the location of a log file

I The presence of this mixin allows cross-providerinteroperability

I As an alternative, the modality can be left implicit(default)

Page 76: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

A monitoring-aware Resource

I A Resource that is the target of a monitoring activitymay be explicitely characterized with a CollectorEnd-Point mixin

I Such a mixin contains the description of the monitoringmodality for that resource

I for instance, the location of a log file

I The presence of this mixin allows cross-providerinteroperability

I As an alternative, the modality can be left implicit(default)

New! This feature is not included in the available revision ofthe document

Page 77: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The overall picture

I Two entity kindsI Sensor aggregates and delivers measurementsI Collector acquires measurements

I Four mixin typesI Aggregator mixins describe the aggregation activity of

a SensorI Publisher mixins describe the rendering activity of a

SensorI Metric mixins describe the measurement activity of a

CollectorI Collector Endpoint mixins describe the monitoring

access point of a target resource

I The two Kinds have a OCCI schema associatedI ...they are defined in the standard

I The Mixins may be associated with a provider specificschema

I ...but we do not exclude that some of them may be partof another standard

Page 78: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The overall picture

I Two entity kindsI Sensor aggregates and delivers measurementsI Collector acquires measurements

I Four mixin typesI Aggregator mixins describe the aggregation activity of

a SensorI Publisher mixins describe the rendering activity of a

SensorI Metric mixins describe the measurement activity of a

CollectorI Collector Endpoint mixins describe the monitoring

access point of a target resource

I The two Kinds have a OCCI schema associatedI ...they are defined in the standard

I The Mixins may be associated with a provider specificschema

I ...but we do not exclude that some of them may be partof another standard

Page 79: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The overall picture

I Two entity kindsI Sensor aggregates and delivers measurementsI Collector acquires measurements

I Four mixin typesI Aggregator mixins describe the aggregation activity of

a SensorI Publisher mixins describe the rendering activity of a

SensorI Metric mixins describe the measurement activity of a

CollectorI Collector Endpoint mixins describe the monitoring

access point of a target resource

I The two Kinds have a OCCI schema associatedI ...they are defined in the standard

I The Mixins may be associated with a provider specificschema

I ...but we do not exclude that some of them may be partof another standard

Page 80: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The overall picture

I Two entity kindsI Sensor aggregates and delivers measurementsI Collector acquires measurements

I Four mixin typesI Aggregator mixins describe the aggregation activity of

a SensorI Publisher mixins describe the rendering activity of a

SensorI Metric mixins describe the measurement activity of a

CollectorI Collector Endpoint mixins describe the monitoring

access point of a target resource

I The two Kinds have a OCCI schema associatedI ...they are defined in the standard

I The Mixins may be associated with a provider specificschema

I ...but we do not exclude that some of them may be partof another standard

Page 81: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The overall picture

I Two entity kindsI Sensor aggregates and delivers measurementsI Collector acquires measurements

I Four mixin typesI Aggregator mixins describe the aggregation activity of

a SensorI Publisher mixins describe the rendering activity of a

SensorI Metric mixins describe the measurement activity of a

CollectorI Collector Endpoint mixins describe the monitoring

access point of a target resource

I The two Kinds have a OCCI schema associatedI ...they are defined in the standard

I The Mixins may be associated with a provider specificschema

I ...but we do not exclude that some of them may be partof another standard

Page 82: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The overall picture

I Two entity kindsI Sensor aggregates and delivers measurementsI Collector acquires measurements

I Four mixin typesI Aggregator mixins describe the aggregation activity of

a SensorI Publisher mixins describe the rendering activity of a

SensorI Metric mixins describe the measurement activity of a

CollectorI Collector Endpoint mixins describe the monitoring

access point of a target resource

I The two Kinds have a OCCI schema associatedI ...they are defined in the standard

I The Mixins may be associated with a provider specificschema

I ...but we do not exclude that some of them may be partof another standard

Page 83: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The overall picture

I Two entity kindsI Sensor aggregates and delivers measurementsI Collector acquires measurements

I Four mixin typesI Aggregator mixins describe the aggregation activity of

a SensorI Publisher mixins describe the rendering activity of a

SensorI Metric mixins describe the measurement activity of a

CollectorI Collector Endpoint mixins describe the monitoring

access point of a target resource

I The two Kinds have a OCCI schema associatedI ...they are defined in the standard

I The Mixins may be associated with a provider specificschema

I ...but we do not exclude that some of them may be partof another standard

Page 84: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The overall picture

I Two entity kindsI Sensor aggregates and delivers measurementsI Collector acquires measurements

I Four mixin typesI Aggregator mixins describe the aggregation activity of

a SensorI Publisher mixins describe the rendering activity of a

SensorI Metric mixins describe the measurement activity of a

CollectorI Collector Endpoint mixins describe the monitoring

access point of a target resource

I The two Kinds have a OCCI schema associatedI ...they are defined in the standard

I The Mixins may be associated with a provider specificschema

I ...but we do not exclude that some of them may be partof another standard

Page 85: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The overall picture

I Two entity kindsI Sensor aggregates and delivers measurementsI Collector acquires measurements

I Four mixin typesI Aggregator mixins describe the aggregation activity of

a SensorI Publisher mixins describe the rendering activity of a

SensorI Metric mixins describe the measurement activity of a

CollectorI Collector Endpoint mixins describe the monitoring

access point of a target resource

I The two Kinds have a OCCI schema associatedI ...they are defined in the standard

I The Mixins may be associated with a provider specificschema

I ...but we do not exclude that some of them may be partof another standard

Page 86: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The overall picture

I Two entity kindsI Sensor aggregates and delivers measurementsI Collector acquires measurements

I Four mixin typesI Aggregator mixins describe the aggregation activity of

a SensorI Publisher mixins describe the rendering activity of a

SensorI Metric mixins describe the measurement activity of a

CollectorI Collector Endpoint mixins describe the monitoring

access point of a target resource

I The two Kinds have a OCCI schema associatedI ...they are defined in the standard

I The Mixins may be associated with a provider specificschema

I ...but we do not exclude that some of them may be partof another standard

Page 87: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The overall picture

I Two entity kindsI Sensor aggregates and delivers measurementsI Collector acquires measurements

I Four mixin typesI Aggregator mixins describe the aggregation activity of

a SensorI Publisher mixins describe the rendering activity of a

SensorI Metric mixins describe the measurement activity of a

CollectorI Collector Endpoint mixins describe the monitoring

access point of a target resource

I The two Kinds have a OCCI schema associatedI ...they are defined in the standard

I The Mixins may be associated with a provider specificschema

I ...but we do not exclude that some of them may be partof another standard

Page 88: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

The overall picture

I Two entity kindsI Sensor aggregates and delivers measurementsI Collector acquires measurements

I Four mixin typesI Aggregator mixins describe the aggregation activity of

a SensorI Publisher mixins describe the rendering activity of a

SensorI Metric mixins describe the measurement activity of a

CollectorI Collector Endpoint mixins describe the monitoring

access point of a target resource

I The two Kinds have a OCCI schema associatedI ...they are defined in the standard

I The Mixins may be associated with a provider specificschema

I ...but we do not exclude that some of them may be partof another standard

Page 89: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Hold them together: input and output hooks

I The designer needs a tool to assemble a monitoringinfrastructure

I we introduce input and output attributes for the Mixins

I We introduce hook attributes (named attributes in thelast revision) for a mixin:

I their value corresponds to a labelI Input attributesI Output attributes

I Input and Output hooks with matching labels areconnected

I this may mean a data flow among them

I The scope of hook labels is limited to a sensor and itsadjacent collectors

I The provider indicates hook semantics in thespecifications of the mixin

Page 90: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Hold them together: input and output hooks

I The designer needs a tool to assemble a monitoringinfrastructure

I we introduce input and output attributes for the Mixins

I We introduce hook attributes (named attributes in thelast revision) for a mixin:

I their value corresponds to a labelI Input attributesI Output attributes

I Input and Output hooks with matching labels areconnected

I this may mean a data flow among them

I The scope of hook labels is limited to a sensor and itsadjacent collectors

I The provider indicates hook semantics in thespecifications of the mixin

Page 91: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Hold them together: input and output hooks

I The designer needs a tool to assemble a monitoringinfrastructure

I we introduce input and output attributes for the Mixins

I We introduce hook attributes (named attributes in thelast revision) for a mixin:

I their value corresponds to a labelI Input attributesI Output attributes

I Input and Output hooks with matching labels areconnected

I this may mean a data flow among them

I The scope of hook labels is limited to a sensor and itsadjacent collectors

I The provider indicates hook semantics in thespecifications of the mixin

Page 92: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Hold them together: input and output hooks

I The designer needs a tool to assemble a monitoringinfrastructure

I we introduce input and output attributes for the Mixins

I We introduce hook attributes (named attributes in thelast revision) for a mixin:

I their value corresponds to a labelI Input attributesI Output attributes

I Input and Output hooks with matching labels areconnected

I this may mean a data flow among them

I The scope of hook labels is limited to a sensor and itsadjacent collectors

I The provider indicates hook semantics in thespecifications of the mixin

Page 93: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Hold them together: input and output hooks

I The designer needs a tool to assemble a monitoringinfrastructure

I we introduce input and output attributes for the Mixins

I We introduce hook attributes (named attributes in thelast revision) for a mixin:

I their value corresponds to a labelI Input attributesI Output attributes

I Input and Output hooks with matching labels areconnected

I this may mean a data flow among them

I The scope of hook labels is limited to a sensor and itsadjacent collectors

I The provider indicates hook semantics in thespecifications of the mixin

Page 94: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Hold them together: input and output hooks

I The designer needs a tool to assemble a monitoringinfrastructure

I we introduce input and output attributes for the Mixins

I We introduce hook attributes (named attributes in thelast revision) for a mixin:

I their value corresponds to a labelI Input attributesI Output attributes

I Input and Output hooks with matching labels areconnected

I this may mean a data flow among them

I The scope of hook labels is limited to a sensor and itsadjacent collectors

I The provider indicates hook semantics in thespecifications of the mixin

Page 95: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Hold them together: input and output hooks

I The designer needs a tool to assemble a monitoringinfrastructure

I we introduce input and output attributes for the Mixins

I We introduce hook attributes (named attributes in thelast revision) for a mixin:

I their value corresponds to a labelI Input attributesI Output attributes

I Input and Output hooks with matching labels areconnected

I this may mean a data flow among them

I The scope of hook labels is limited to a sensor and itsadjacent collectors

I The provider indicates hook semantics in thespecifications of the mixin

Page 96: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Hold them together: input and output hooks

I The designer needs a tool to assemble a monitoringinfrastructure

I we introduce input and output attributes for the Mixins

I We introduce hook attributes (named attributes in thelast revision) for a mixin:

I their value corresponds to a labelI Input attributesI Output attributes

I Input and Output hooks with matching labels areconnected

I this may mean a data flow among them

I The scope of hook labels is limited to a sensor and itsadjacent collectors

I The provider indicates hook semantics in thespecifications of the mixin

Page 97: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Hold them together: input and output hooks

I The designer needs a tool to assemble a monitoringinfrastructure

I we introduce input and output attributes for the Mixins

I We introduce hook attributes (named attributes in thelast revision) for a mixin:

I their value corresponds to a labelI Input attributesI Output attributes

I Input and Output hooks with matching labels areconnected

I this may mean a data flow among them

I The scope of hook labels is limited to a sensor and itsadjacent collectors

I The provider indicates hook semantics in thespecifications of the mixin

Page 98: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Hold them together: input and output hooks

I The designer needs a tool to assemble a monitoringinfrastructure

I we introduce input and output attributes for the Mixins

I We introduce hook attributes (named attributes in thelast revision) for a mixin:

I their value corresponds to a labelI Input attributesI Output attributes

I Input and Output hooks with matching labels areconnected

I this may mean a data flow among them

I The scope of hook labels is limited to a sensor and itsadjacent collectors

I The provider indicates hook semantics in thespecifications of the mixin

Page 99: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

An example:

I One sensor collects measurementes from two resources

I Results are published through two different channels(e.g., streaming and database)

I Two distinct measurement tools are applied to each ofthe two resources (total four tools)

I We combine a metric from both resources (e.g., averageload)

Page 100: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

An example:

I One sensor collects measurementes from two resources

I Results are published through two different channels(e.g., streaming and database)

I Two distinct measurement tools are applied to each ofthe two resources (total four tools)

I We combine a metric from both resources (e.g., averageload)

Page 101: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

An example:

I One sensor collects measurementes from two resources

I Results are published through two different channels(e.g., streaming and database)

I Two distinct measurement tools are applied to each ofthe two resources (total four tools)

I We combine a metric from both resources (e.g., averageload)

Page 102: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

An example:

I One sensor collects measurementes from two resources

I Results are published through two different channels(e.g., streaming and database)

I Two distinct measurement tools are applied to each ofthe two resources (total four tools)

I We combine a metric from both resources (e.g., averageload)

Page 103: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

An example:

I One sensor collects measurementes from two resources

I Results are published through two different channels(e.g., streaming and database)

I Two distinct measurement tools are applied to each ofthe two resources (total four tools)

I We combine a metric from both resources (e.g., averageload)

Page 104: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

The resources we want to monitor: they have a CollectorEndpoint mixin associated

Page 105: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

Create one Sensor resource

Page 106: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

Use two collectors to define the measurement activiy

Page 107: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

Associate two metric mixins to the Collector X

Page 108: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

And another two metric mixins to the Collector Y

Page 109: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

Associate two aggregator mixins to the Sensor

Page 110: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

One publisher is going to use raw data from the collector

Page 111: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

Another is going to receive measurements from theaggregators

Page 112: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

A frame for Collector X and its mixins

Page 113: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

... one for Collector Y...

Page 114: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

... one for the sensor

Page 115: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

The scope of the Sensor (for hook labels)

Page 116: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

Feeding the aggregators: a,b,d are hook labels

Page 117: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

Feeding publisher 2: aggregated (f,g) and raw data (e)

Page 118: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

Step by step design of a monitoring infrastructure

Feeding publisher 1: measurement stream b is multicast

Page 119: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

ConclusionsI To manage cloud resource, we need to monitor them

I indeed many providers offer monitoring as a serviceI We establish a minimum common ground for

interoperabilityI a standard, aligned with the Open Cloud Computing

InterfaceI Two entities:

I Collector link to produce monitoring dataI Sensor resource to aggregate and deliver monitoring

dataI Finalized using mixins defined by the providerI Can be combined to form complex monitoring

infrastructuresI More... may be extended to any computational

infrastructure

Page 120: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

ConclusionsI To manage cloud resource, we need to monitor them

I indeed many providers offer monitoring as a serviceI We establish a minimum common ground for

interoperabilityI a standard, aligned with the Open Cloud Computing

InterfaceI Two entities:

I Collector link to produce monitoring dataI Sensor resource to aggregate and deliver monitoring

dataI Finalized using mixins defined by the providerI Can be combined to form complex monitoring

infrastructuresI More... may be extended to any computational

infrastructure

Page 121: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

ConclusionsI To manage cloud resource, we need to monitor them

I indeed many providers offer monitoring as a serviceI We establish a minimum common ground for

interoperabilityI a standard, aligned with the Open Cloud Computing

InterfaceI Two entities:

I Collector link to produce monitoring dataI Sensor resource to aggregate and deliver monitoring

dataI Finalized using mixins defined by the providerI Can be combined to form complex monitoring

infrastructuresI More... may be extended to any computational

infrastructure

Page 122: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

ConclusionsI To manage cloud resource, we need to monitor them

I indeed many providers offer monitoring as a serviceI We establish a minimum common ground for

interoperabilityI a standard, aligned with the Open Cloud Computing

InterfaceI Two entities:

I Collector link to produce monitoring dataI Sensor resource to aggregate and deliver monitoring

dataI Finalized using mixins defined by the providerI Can be combined to form complex monitoring

infrastructuresI More... may be extended to any computational

infrastructure

Page 123: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

ConclusionsI To manage cloud resource, we need to monitor them

I indeed many providers offer monitoring as a serviceI We establish a minimum common ground for

interoperabilityI a standard, aligned with the Open Cloud Computing

InterfaceI Two entities:

I Collector link to produce monitoring dataI Sensor resource to aggregate and deliver monitoring

dataI Finalized using mixins defined by the providerI Can be combined to form complex monitoring

infrastructuresI More... may be extended to any computational

infrastructure

Page 124: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

ConclusionsI To manage cloud resource, we need to monitor them

I indeed many providers offer monitoring as a serviceI We establish a minimum common ground for

interoperabilityI a standard, aligned with the Open Cloud Computing

InterfaceI Two entities:

I Collector link to produce monitoring dataI Sensor resource to aggregate and deliver monitoring

dataI Finalized using mixins defined by the providerI Can be combined to form complex monitoring

infrastructuresI More... may be extended to any computational

infrastructure

Page 125: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

ConclusionsI To manage cloud resource, we need to monitor them

I indeed many providers offer monitoring as a serviceI We establish a minimum common ground for

interoperabilityI a standard, aligned with the Open Cloud Computing

InterfaceI Two entities:

I Collector link to produce monitoring dataI Sensor resource to aggregate and deliver monitoring

dataI Finalized using mixins defined by the providerI Can be combined to form complex monitoring

infrastructuresI More... may be extended to any computational

infrastructure

Page 126: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

ConclusionsI To manage cloud resource, we need to monitor them

I indeed many providers offer monitoring as a serviceI We establish a minimum common ground for

interoperabilityI a standard, aligned with the Open Cloud Computing

InterfaceI Two entities:

I Collector link to produce monitoring dataI Sensor resource to aggregate and deliver monitoring

dataI Finalized using mixins defined by the providerI Can be combined to form complex monitoring

infrastructuresI More... may be extended to any computational

infrastructure

Page 127: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

ConclusionsI To manage cloud resource, we need to monitor them

I indeed many providers offer monitoring as a serviceI We establish a minimum common ground for

interoperabilityI a standard, aligned with the Open Cloud Computing

InterfaceI Two entities:

I Collector link to produce monitoring dataI Sensor resource to aggregate and deliver monitoring

dataI Finalized using mixins defined by the providerI Can be combined to form complex monitoring

infrastructuresI More... may be extended to any computational

infrastructure

Page 128: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

ConclusionsI To manage cloud resource, we need to monitor them

I indeed many providers offer monitoring as a serviceI We establish a minimum common ground for

interoperabilityI a standard, aligned with the Open Cloud Computing

InterfaceI Two entities:

I Collector link to produce monitoring dataI Sensor resource to aggregate and deliver monitoring

dataI Finalized using mixins defined by the providerI Can be combined to form complex monitoring

infrastructuresI More... may be extended to any computational

infrastructure

Page 129: OCCI Monitoring at OGF42 - Concepts and demo

OCCI monitoringextension

Augusto Ciuffoletti

ConclusionsI To manage cloud resource, we need to monitor them

I indeed many providers offer monitoring as a serviceI We establish a minimum common ground for

interoperabilityI a standard, aligned with the Open Cloud Computing

InterfaceI Two entities:

I Collector link to produce monitoring dataI Sensor resource to aggregate and deliver monitoring

dataI Finalized using mixins defined by the providerI Can be combined to form complex monitoring

infrastructuresI More... may be extended to any computational

infrastructure

End of part 1

Page 130: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

From the OCCI to Java: unfolding theinfrastructure

A Java backend for OCCI monitoring

Augusto Ciuffoletti

Dept. of Computer Science - Univ. of Pisa

September 5, 2014

Page 131: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

An experiment to verify an abstract specification

The framework

I Limited to the backend, in the perspective of theprovider that implements the monitoring service

I No user interface: OCCI-JSON documents are alreadyon the web server

I Written in Java, because it is widely understood

The question - Is the specification realistic?

I is the sensor+collector model sufficiently descriptive?

I are the attributes enough to describe a monitoringframework?

I are mixins modular with respect to the framework?

I is there a simple way to implement it efficiently?

The answers are basically positive, but something interestinghas been discovered...

Page 132: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

An experiment to verify an abstract specification

The framework

I Limited to the backend, in the perspective of theprovider that implements the monitoring service

I No user interface: OCCI-JSON documents are alreadyon the web server

I Written in Java, because it is widely understood

The question - Is the specification realistic?

I is the sensor+collector model sufficiently descriptive?

I are the attributes enough to describe a monitoringframework?

I are mixins modular with respect to the framework?

I is there a simple way to implement it efficiently?

The answers are basically positive, but something interestinghas been discovered...

Page 133: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

An experiment to verify an abstract specification

The framework

I Limited to the backend, in the perspective of theprovider that implements the monitoring service

I No user interface: OCCI-JSON documents are alreadyon the web server

I Written in Java, because it is widely understood

The question - Is the specification realistic?

I is the sensor+collector model sufficiently descriptive?

I are the attributes enough to describe a monitoringframework?

I are mixins modular with respect to the framework?

I is there a simple way to implement it efficiently?

The answers are basically positive, but something interestinghas been discovered...

Page 134: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

An experiment to verify an abstract specification

The framework

I Limited to the backend, in the perspective of theprovider that implements the monitoring service

I No user interface: OCCI-JSON documents are alreadyon the web server

I Written in Java, because it is widely understood

The question - Is the specification realistic?

I is the sensor+collector model sufficiently descriptive?

I are the attributes enough to describe a monitoringframework?

I are mixins modular with respect to the framework?

I is there a simple way to implement it efficiently?

The answers are basically positive, but something interestinghas been discovered...

Page 135: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

An experiment to verify an abstract specification

The framework

I Limited to the backend, in the perspective of theprovider that implements the monitoring service

I No user interface: OCCI-JSON documents are alreadyon the web server

I Written in Java, because it is widely understood

The question - Is the specification realistic?

I is the sensor+collector model sufficiently descriptive?

I are the attributes enough to describe a monitoringframework?

I are mixins modular with respect to the framework?

I is there a simple way to implement it efficiently?

The answers are basically positive, but something interestinghas been discovered...

Page 136: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

An experiment to verify an abstract specification

The framework

I Limited to the backend, in the perspective of theprovider that implements the monitoring service

I No user interface: OCCI-JSON documents are alreadyon the web server

I Written in Java, because it is widely understood

The question - Is the specification realistic?

I is the sensor+collector model sufficiently descriptive?

I are the attributes enough to describe a monitoringframework?

I are mixins modular with respect to the framework?

I is there a simple way to implement it efficiently?

The answers are basically positive, but something interestinghas been discovered...

Page 137: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

An experiment to verify an abstract specification

The framework

I Limited to the backend, in the perspective of theprovider that implements the monitoring service

I No user interface: OCCI-JSON documents are alreadyon the web server

I Written in Java, because it is widely understood

The question - Is the specification realistic?

I is the sensor+collector model sufficiently descriptive?

I are the attributes enough to describe a monitoringframework?

I are mixins modular with respect to the framework?

I is there a simple way to implement it efficiently?

The answers are basically positive, but something interestinghas been discovered...

Page 138: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

An experiment to verify an abstract specification

The framework

I Limited to the backend, in the perspective of theprovider that implements the monitoring service

I No user interface: OCCI-JSON documents are alreadyon the web server

I Written in Java, because it is widely understood

The question - Is the specification realistic?

I is the sensor+collector model sufficiently descriptive?

I are the attributes enough to describe a monitoringframework?

I are mixins modular with respect to the framework?

I is there a simple way to implement it efficiently?

The answers are basically positive, but something interestinghas been discovered...

Page 139: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

An experiment to verify an abstract specification

The framework

I Limited to the backend, in the perspective of theprovider that implements the monitoring service

I No user interface: OCCI-JSON documents are alreadyon the web server

I Written in Java, because it is widely understood

The question - Is the specification realistic?

I is the sensor+collector model sufficiently descriptive?

I are the attributes enough to describe a monitoringframework?

I are mixins modular with respect to the framework?

I is there a simple way to implement it efficiently?

The answers are basically positive, but something interestinghas been discovered...

Page 140: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

An experiment to verify an abstract specification

The framework

I Limited to the backend, in the perspective of theprovider that implements the monitoring service

I No user interface: OCCI-JSON documents are alreadyon the web server

I Written in Java, because it is widely understood

The question - Is the specification realistic?

I is the sensor+collector model sufficiently descriptive?

I are the attributes enough to describe a monitoringframework?

I are mixins modular with respect to the framework?

I is there a simple way to implement it efficiently?

The answers are basically positive, but something interestinghas been discovered...

Page 141: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Basic design options

I The Collector Endpoint mixin is implemented on acompute resource as a daemon with an RMI interface

I Metric mixins are dynamically created threads (Javareflection)

I Metric classes are not dynamically loaded (todo)

I The Sensor runs on a distinct host, possibly sharedwith other Sensors

I Data flow from the Collector Endpoint to the Sensoruses a TCP channel with JSON encoding

I Aggregator and Publisher mixins are dynamicallycreated threads (as for Metrics)

I communication between Aggregators and Publishersuses internal pipes and JSON

I input and output hook attributes are respectivelyassociated with PipedInputStreams and PrintWriters

Page 142: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Basic design options

I The Collector Endpoint mixin is implemented on acompute resource as a daemon with an RMI interface

I Metric mixins are dynamically created threads (Javareflection)

I Metric classes are not dynamically loaded (todo)

I The Sensor runs on a distinct host, possibly sharedwith other Sensors

I Data flow from the Collector Endpoint to the Sensoruses a TCP channel with JSON encoding

I Aggregator and Publisher mixins are dynamicallycreated threads (as for Metrics)

I communication between Aggregators and Publishersuses internal pipes and JSON

I input and output hook attributes are respectivelyassociated with PipedInputStreams and PrintWriters

Page 143: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Basic design options

I The Collector Endpoint mixin is implemented on acompute resource as a daemon with an RMI interface

I Metric mixins are dynamically created threads (Javareflection)

I Metric classes are not dynamically loaded (todo)

I The Sensor runs on a distinct host, possibly sharedwith other Sensors

I Data flow from the Collector Endpoint to the Sensoruses a TCP channel with JSON encoding

I Aggregator and Publisher mixins are dynamicallycreated threads (as for Metrics)

I communication between Aggregators and Publishersuses internal pipes and JSON

I input and output hook attributes are respectivelyassociated with PipedInputStreams and PrintWriters

Page 144: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Basic design options

I The Collector Endpoint mixin is implemented on acompute resource as a daemon with an RMI interface

I Metric mixins are dynamically created threads (Javareflection)

I Metric classes are not dynamically loaded (todo)

I The Sensor runs on a distinct host, possibly sharedwith other Sensors

I Data flow from the Collector Endpoint to the Sensoruses a TCP channel with JSON encoding

I Aggregator and Publisher mixins are dynamicallycreated threads (as for Metrics)

I communication between Aggregators and Publishersuses internal pipes and JSON

I input and output hook attributes are respectivelyassociated with PipedInputStreams and PrintWriters

Page 145: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Basic design options

I The Collector Endpoint mixin is implemented on acompute resource as a daemon with an RMI interface

I Metric mixins are dynamically created threads (Javareflection)

I Metric classes are not dynamically loaded (todo)

I The Sensor runs on a distinct host, possibly sharedwith other Sensors

I Data flow from the Collector Endpoint to the Sensoruses a TCP channel with JSON encoding

I Aggregator and Publisher mixins are dynamicallycreated threads (as for Metrics)

I communication between Aggregators and Publishersuses internal pipes and JSON

I input and output hook attributes are respectivelyassociated with PipedInputStreams and PrintWriters

Page 146: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Basic design options

I The Collector Endpoint mixin is implemented on acompute resource as a daemon with an RMI interface

I Metric mixins are dynamically created threads (Javareflection)

I Metric classes are not dynamically loaded (todo)

I The Sensor runs on a distinct host, possibly sharedwith other Sensors

I Data flow from the Collector Endpoint to the Sensoruses a TCP channel with JSON encoding

I Aggregator and Publisher mixins are dynamicallycreated threads (as for Metrics)

I communication between Aggregators and Publishersuses internal pipes and JSON

I input and output hook attributes are respectivelyassociated with PipedInputStreams and PrintWriters

Page 147: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Basic design options

I The Collector Endpoint mixin is implemented on acompute resource as a daemon with an RMI interface

I Metric mixins are dynamically created threads (Javareflection)

I Metric classes are not dynamically loaded (todo)

I The Sensor runs on a distinct host, possibly sharedwith other Sensors

I Data flow from the Collector Endpoint to the Sensoruses a TCP channel with JSON encoding

I Aggregator and Publisher mixins are dynamicallycreated threads (as for Metrics)

I communication between Aggregators and Publishersuses internal pipes and JSON

I input and output hook attributes are respectivelyassociated with PipedInputStreams and PrintWriters

Page 148: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Basic design options

I The Collector Endpoint mixin is implemented on acompute resource as a daemon with an RMI interface

I Metric mixins are dynamically created threads (Javareflection)

I Metric classes are not dynamically loaded (todo)

I The Sensor runs on a distinct host, possibly sharedwith other Sensors

I Data flow from the Collector Endpoint to the Sensoruses a TCP channel with JSON encoding

I Aggregator and Publisher mixins are dynamicallycreated threads (as for Metrics)

I communication between Aggregators and Publishersuses internal pipes and JSON

I input and output hook attributes are respectivelyassociated with PipedInputStreams and PrintWriters

Page 149: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

The tools

I Java 7: either Oracle or OpenJDK

I json-simple 1.1.1

JSON.simple is a simple Java toolkit forJSON. You can use JSON.simple to encode ordecode JSON text.

I jsoup 1.7.3

jsoup is a Java library for working withreal-world HTML. It provides a veryconvenient API for extracting andmanipulating data, using the best of DOM,CSS, and jquery-like methods.

Page 150: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

The tools

I Java 7: either Oracle or OpenJDK

I json-simple 1.1.1

JSON.simple is a simple Java toolkit forJSON. You can use JSON.simple to encode ordecode JSON text.

I jsoup 1.7.3

jsoup is a Java library for working withreal-world HTML. It provides a veryconvenient API for extracting andmanipulating data, using the best of DOM,CSS, and jquery-like methods.

Page 151: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

The tools

I Java 7: either Oracle or OpenJDK

I json-simple 1.1.1

JSON.simple is a simple Java toolkit forJSON. You can use JSON.simple to encode ordecode JSON text.

I jsoup 1.7.3

jsoup is a Java library for working withreal-world HTML. It provides a veryconvenient API for extracting andmanipulating data, using the best of DOM,CSS, and jquery-like methods.

Page 152: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

The tools

I Java 7: either Oracle or OpenJDK

I json-simple 1.1.1

JSON.simple is a simple Java toolkit forJSON. You can use JSON.simple to encode ordecode JSON text.

I jsoup 1.7.3

jsoup is a Java library for working withreal-world HTML. It provides a veryconvenient API for extracting andmanipulating data, using the best of DOM,CSS, and jquery-like methods.

Page 153: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

The tools

I Java 7: either Oracle or OpenJDK

I json-simple 1.1.1

JSON.simple is a simple Java toolkit forJSON. You can use JSON.simple to encode ordecode JSON text.

I jsoup 1.7.3

jsoup is a Java library for working withreal-world HTML. It provides a veryconvenient API for extracting andmanipulating data, using the best of DOM,CSS, and jquery-like methods.

Page 154: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

An example: monitoring load and connectivity ofa Compute resource

I The measurement of the average CPU load is sentoutside the cloud as a UDP flow

I The connectivity with the gateway is collected in a logfile on the sensor.

Page 155: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

An example: The Collector

{

"id": "urn:uuid:2345",

"kind": "http://schemas.ogf.org/occi/monitoring#collector",

"mixins": [

"http://example.com/occi/monitoring/metric#CPUPercent"

"http://example.com/occi/monitoring/metric#isReachable"

],

"attributes": {"occi": {"collector": {

"period": 60

}},

"com": {"example": {"occi": { "monitoring": {

"CPUPercent" : {"out": "a"},

"IsReachable" : {"hostname": "192.168.5.1","maxdelay": 1000,

"out":"b"}

}}}}},

"actions": [],

"target":"urn:uuid:s1111",

"source":"urn:uuid:c2222"

}

Page 156: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

An example: The Sensor

{

"id": "urn:uuid:s1111",

"kind": "http://schemas.ogf.org/occi/monitoring#sensor",

"mixins": [

"http://example.com/occi/monitoring/publisher#SendUDP",

"http://example.com/occi/monitoring/aggregator#EWMA"

"http://example.com/occi/monitoring/publisher#Log

],

"attributes": { "occi": { "sensor": {

"period": 60, "timebase": 1386925386,

"timestart": 600, "timestop": 3600,

"networkInterface": "eth0"}

},

"com": {"example": {"occi": {"monitoring": {

"SendUDP" : {"udpAddr": "192.168.5.2:8888","input": "c"},

"EWMA" : {"gain": 16,"instream": "a","outstream": "c"},

"Log" : {"filename": "my/log/file","in_msg": "b"}

}}}}},

"links": ["urn:uuid:2345"]

}

Page 157: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

An example: The target Resource

{

"id": "urn:uuid:c2222",

"kind": "http://schemas.ogf.org/occi/infrastructure#compute",

"mixins": [

"http.//schemas.ogf.org/infrastructure/compute#RMIMetricContainer"

],

"attributes": { "occi": { "compute":

{ "speed": 2, "memory": 4, "cores": 2,

"MetricContainerIPAddress": "192.168.5.3",

"MetricContainerPort": 12312 }}},

"actions": []

}

Page 158: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Unfolding the sensor

I The sensor starts as a tread in a thread pool

I It reads its specifications using an HTTP GET

I A TCP socket (server side) is allocated for input fromthe collectors

Page 159: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Unfolding the sensor

I The sensor launches a Collector Manager thread foreach collector in the scope

Page 160: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Unfolding the sensor

I The Collector Manager invokes (by RMI) themeasurement threads on the source

I Each of them opens a TCP connection with the sensorsocket

Page 161: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Unfolding the sensor

I The Collector Manager creates the pipes used forcommunication

I There is one pipe for each hook label in the scope

I Records from the socket are multiplexed to pipesaccording with hook identifiers

Page 162: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Unfolding the sensor

I Create the threads that implement the Aggregators

I Input and output hooks are bound to pipes

Page 163: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Unfolding the sensor

I Create the threads that implement the Publishers

I Input and output channels are bound as for Aggregators

Page 164: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Unfolding the sensor

I One publisher is going to use raw data from thecollector

Page 165: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Unfolding the Collector Endpoint

I The monitored resource runs a daemon with an RMIinterface (MetricContainer)

I The sensor has a TCP server-side port acceptingconnections

I The sensor learns the RMI interface from OCCIdocuments

Page 166: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Unfolding the Collector Endpoint

I The CollectorManager on the sensor calls a remoteLaunchMetric method on the metric container

I The parameters includeI the name of the metric mixinI the attributes of the mixin, encoded as a JSON object

Page 167: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Unfolding the Collector Endpoint

I The MetricContainer creates a metric thread (usingreflection)

I The metric thread opens a connection to the Collectorsocket on the Sensor

Page 168: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Unfolding the Collector Endpoint

I All metrics associated with a Collector share the sameTCP connection to the Sensor

Page 169: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Unfolding the Collector Endpoint

I The messages from the metric to the Sensor are JSONdocuments, tagged with the destination hook

Page 170: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Finally, the experiment

I A virtual network with guest + 3VM

I One VM acts as “coordinator”, in fact simply holdingthe OCCI specs as application/occi+json documents ina web server

I One VM acts as monitored resource

I One VM acts as sensor container

I The guest receives measurements through UDP

Page 171: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

On the sensor

Launched by an external manager that allocates sensors todedicated hosts

Page 172: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

On the monitored compute resource

Launched as a consequence of the presence of aMetricContainer mixin

Page 173: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

On the web server

I the CollectorEndpoint starts first

I the Sensor downloads the documents that describe itsscope (caching!)

Page 174: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Conclusions of the experiment

I The proof of concept still needs some work to befinished (as a proof of concept)

I The part of the specifications concerning “named”attributes (now hooks) has been validated

I The Java implementation can be useful as a blueprintfor a real implementation

Page 175: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Overall conclusions (document update)

I The MetricContainer added as a SHOULD(recommended, not strictly needed)

I Change terminology for the “named attribute” (intohook, or channel)

Page 176: OCCI Monitoring at OGF42 - Concepts and demo

From the OCCI toJava: unfolding

the infrastructure

Augusto Ciuffoletti

Overall conclusions (document update)

I The MetricContainer added as a SHOULD(recommended, not strictly needed)

I Change terminology for the “named attribute” (intohook, or channel)