Federated DCs Migration Controller - Project Catalyst · 2020. 4. 15. · d3.3 federated dcs...

D3.3

Federated DCs Migration

Controller

WORKPACKAGE

WP3

DOCUMENT

D3.3

VERSION

1.0

PUBLISH DATE

27/09/2019

DOCUMENT REFERENCE

CATALYST.D3.3.SiLO.WP3.v1.0

PROGRAMME IDENTIFIER

H2020-EE-2016-2017

PROJECT NUMBER

768739

START DATE OF THE PROJECT

01/10/2017

DURATION

36 months

CATALYST.D3.3.SiLO.WP3.v1.0 H2020-EE-2016-2017

Federated DCs Migration Controller 768739

1

PROGRAMME NAME ENERGY EFFICIENCY CALL 2016-2017

PROGRAMME IDENTIFIER H2020-EE-2016-2017

TOPIC Bringing to market more energy efficient and integrated data centres

TOPIC IDENTIFIER EE-20-2017

TYPE OF ACTION IA Innovation action

PROJECT NUMBER 768739

PROJECT TITLE CATALYST

COORDINATOR ENGINEERING INGEGNERIA INFORMATICA S.p.A. (ENG)

PRINCIPAL CONTRACTORS SINGULARLOGIC ANONYMI ETAIREIA PLIROFORIAKON SYSTIMATON KAI

EFARMOGON PLIROFORIKIS (SiLO), ENEL.SI S.r.l (ENEL), ALLIANDER NV

(ALD), STICHTING GREEN IT CONSORTIUM REGIO AMSTERDAM (GIT),

SCHUBERG PHILIS BV (SBP), QARNOT COMPUTING (QRN), POWER

OPERATIONS LIMITED (POPs), INSTYTUT CHEMII BIOORGANICZNEJ

POLSKIEJ AKADEMII NAUK (PSNC), UNIVERSITATEA TEHNICA CLUJ-NAPOCA

(TUC)

DOCUMENT REFERENCE CATALYST.D3.3.SiLO.WP3.v1.0

WORKPACKAGE: WP3

DELIVERABLE TYPE OTHER

AVAILABILITY PU

DELIVERABLE STATE Final

CONTRACTUAL DATE OF DELIVERY 30/09/2019

ACTUAL DATE OF DELIVERY 27/09/2019

DOCUMENT TITLE Federated DCs Migration Controller

AUTHOR(S) T. Velivassaki (SiLO), A. Voulkidis (POPs)

REVIEWER(S) M. Mammina (ENG), T. Cioara (TUC), I. Anghel (TUC)

SUMMARY (See the Executive Summary)

HISTORY (See the Change History Table)

KEYWORDS Load Migration; SLA Management



2

Change History Version Date State Author (Partner) Description

0.1 01/07/2019 Table of Contents

(ToC)

T. Velivassaki (SiLO) Table of Contents

0.2 12/07/2019 Table of Contents

(ToC)

T. Velivassaki (SiLO) Updates on ToC, based on peer-

reviewers’ feedback

0.3 17/07/2019 Draft T. Velivassaki (SiLO)

A. Voulkidis (POPs)

Elaboration of the Federated DC

Migration Concept


A. Voulkidis (POPs)

Implementation details added


N. Sainthérant (QRN)

Implementation details on Virtual

Containers management and Load

Balancing


A. Voulkidis (POPs)

Additional content on implementation

0.7 07/08/2019 Draft A. Voulkidis (POPs) Additional content on SLA Controller


A. Voulkidis (POPs)

Additional content on implementation

0.8.1 04/09/2019 Draft T. Velivassaki (SiLO)

A. Voulkidis (POPs)

Yoann Ricordel (QRN)

Updating the Federated DC Migration

Concept after QRN comments

0.8.2 11/09/2019 Draft T. Velivassaki (SiLO)

A. Voulkidis (POPs)

Updates on DCMC interfaces

0.9 25/09/2019 Consolidated T. Velivassaki (SiLO) Consolidated report for peer review

0.9.1 26/09/2019 Reviewed M. Mammina (ENG) Peer review comments

0.9.2 26/09/2019 Reviewed I. Anghel (TUC) Peer review comments

0.9.3 27/09/2019 Consolidated T. Velivassaki (SiLO) Consolidated report after peer review

0.9.5 27/09/2019 Release

Candidate

T. Velivassaki (SiLO) Consolidated peer review comments;

Ready to be released

0.9.9 27/09/2019 Quality Checked D. Arnone (ENG) Quality check

1.0 27/09/2019 Final D. Arnone (ENG) PMT approved and ready for

submission to EC



3

Table of Contents Change History ....................................................................................................................................................... 2

Table of Contents ................................................................................................................................................... 3

List of Figures ......................................................................................................................................................... 6

List of Tables .......................................................................................................................................................... 7

List of Acronyms ..................................................................................................................................................... 9

Executive Summary .............................................................................................................................................. 11

1 Introduction ................................................................................................................................................ 12

1.1 Intended Audience .................................................................................................................................. 12

1.2 Relations to other activities .................................................................................................................... 13

1.3 Document overview ................................................................................................................................. 13

2 The Federated DC Migration Concept ...................................................................................................... 14

2.1 The Federated DC Migration as Source of CATALYST Flexibility ........................................................... 14

2.1.1 Identification of loads/resources candidate for migration ................................................................ 15

2.1.2 Trading candidate migrations in the IT Load Marketplace ................................................................ 16

2.1.3 Live migration of loads ......................................................................................................................... 16

2.2 Realizing Federated DC Migration .......................................................................................................... 17

2.2.1 Federated DC Migration Process Design ............................................................................................ 17

3 Implementation .......................................................................................................................................... 20

3.1 Federated DC Migration Controller ........................................................................................................ 20

3.1.1 Architecture ........................................................................................................................................... 20

3.1.1.1 DCMC Server ...................................................................................................................................... 20

3.1.1.2 DCMC Master Client ........................................................................................................................... 21

3.1.1.3 DCMC Lite Client ................................................................................................................................ 22

3.1.2 Interfaces Specification ........................................................................................................................ 22

3.1.2.1 DCMC Server ...................................................................................................................................... 22

3.1.2.1.1 Report accepted migration ............................................................................................................. 22

3.1.2.1.2 Send VC details ................................................................................................................................ 22

3.1.2.1.3 Request token.................................................................................................................................. 23

3.1.2.1.4 Update VC details ............................................................................................................................ 23


3.1.2.2.1 Place a migration request ............................................................................................................... 24

3.1.2.2.2 Associate migration with the CATALYST Marketplace information............................................... 24

3.1.2.2.3 Retrieve migration associated with market action ........................................................................ 25

3.1.2.2.4 Provide migration details ................................................................................................................ 25

3.1.2.2.5 Request VPN creation ..................................................................................................................... 25

3.1.2.2.6 Provide CTL details .......................................................................................................................... 26

3.1.2.2.7 Trigger VC creation .......................................................................................................................... 26



4

3.1.2.2.8 Retrieve transaction associated to VC ........................................................................................... 27

3.1.2.2.9 Inform about VC registration ........................................................................................................... 27

3.1.2.3 DCMC Lite Client ................................................................................................................................ 28

3.1.2.3.1 Receive VPN details ......................................................................................................................... 28

3.1.3 Data Model ............................................................................................................................................ 29

3.1.3.1 DCMC Server ...................................................................................................................................... 29

3.1.3.1.1 Data Model ....................................................................................................................................... 29


3.1.3.2.1 Database Schema ........................................................................................................................... 30

3.1.3.2.2 Data Model ....................................................................................................................................... 30

3.1.3.3 DCMC Lite Client ................................................................................................................................ 32

3.2 SLA (Re-)negotiation ................................................................................................................................ 32

3.2.1 Architecture ........................................................................................................................................... 34


3.2.2.1 Default configuration for new SLAs................................................................................................... 38

3.2.2.2 SLA Events handling .......................................................................................................................... 39

3.2.2.3 SLO levels handling ............................................................................................................................ 41

3.2.2.4 Client notification services ................................................................................................................ 42

3.2.2.5 SLAs handling ..................................................................................................................................... 42

3.2.2.6 SLOs handling .................................................................................................................................... 44

3.2.3 Data Model ............................................................................................................................................ 45

3.3 IT Load Marketplace Connector ............................................................................................................. 48

3.3.1 Architecture ........................................................................................................................................... 48


3.3.2.1 Register candidate migration actions ............................................................................................... 49

3.3.2.2 Register candidate migration actions for specific marketplace timeframe ................................... 49

3.3.2.3 Retrieve reference prices .................................................................................................................. 50

3.3.2.4 Retrieve reference prices for a given date ....................................................................................... 50

3.3.2.5 Register migration status updates .................................................................................................... 51

3.3.3 Data Model ............................................................................................................................................ 51

3.4 Virtual Container Generator .................................................................................................................... 52

3.4.1 Interfaces .............................................................................................................................................. 52

3.5 Energy-aware IT Load Balancing ............................................................................................................ 53

3.5.1 New Interfaces ...................................................................................................................................... 55

3.6 Federated DC Migration Process View ................................................................................................... 55

4 IT Load Balancing and Migration manual ................................................................................................ 58

4.1.1 Virtual Container Generator ................................................................................................................. 58

4.1.2 Energy-aware IT Load Balancer ........................................................................................................... 58

4.1.3 SLA (Re-)negotiation ............................................................................................................................. 58



5

4.1.3.1 Configuration and installation ........................................................................................................... 58

4.1.3.2 SLARC synchronization with existing VCG instances ....................................................................... 60

4.1.4 IT Load Marketplace Connector ........................................................................................................... 61

4.1.5 Federated DC Migration Controller ...................................................................................................... 61

4.1.5.1 DCMC Server ...................................................................................................................................... 61

4.1.5.1.1 Prerequisites .................................................................................................................................... 61

4.1.5.1.2 Deployment ...................................................................................................................................... 62


4.1.5.2.1 Prerequisites .................................................................................................................................... 63

4.1.5.2.2 Deployment ...................................................................................................................................... 63

4.1.5.3 DCMC Lite Client ................................................................................................................................ 63

4.1.5.3.1 Prerequisites .................................................................................................................................... 63

4.1.5.3.2 Deployment ...................................................................................................................................... 63

5 Conclusions ................................................................................................................................................ 64

References ........................................................................................................................................................... 65



6

List of Figures FIGURE 1 - IT LOAD MIGRATION BETWEEN TWO DCS. ................................................................................................... 14

FIGURE 2 - IT LOAD MIGRATION AS REALIZED IN THE CATALYST ARCHITECTURE. ........................................................... 15

FIGURE 3 - PREPARING FOR MIGRATING LOAD FROM DC1 TO DC2. ................................................................................ 17

FIGURE 4 - LIVE MIGRATION OF LOAD FROM DC1 TO DC2. ............................................................................................ 18

FIGURE 5 - THE DCMC ARCHITECTURE. ..................................................................................................................... 20

FIGURE 6 - THE DCMC MASTER CLIENT DATABASE SCHEMA. ....................................................................................... 30

FIGURE 7 - CONCEPTUAL BASE SLARC DATA MODEL ARCHITECTURE. ............................................................................ 33

FIGURE 8 - HANDLING NEW EXTERNAL EVENTS............................................................................................................ 34

FIGURE 9 - HANDLING THE FINALIZATION OF EXTERNAL EVENTS. ................................................................................... 34

FIGURE 10 - THE SLA (RE-)NEGOTIATION CONTROLLER ARCHITECTURE. ........................................................................ 35

FIGURE 11 - SLARC NOTIFYING FOR SLA BREAKAGE AN EXTERNAL KAFKA INTERFACE.................................................... 36

FIGURE 12 - SLARC NOTIFYING FOR SLA BREAKAGE THE INTERNAL TASKS QUEUE MANAGER......................................... 37

FIGURE 13 - SLARC HIGH-LEVEL API SPECIFICATION. ................................................................................................. 37

FIGURE 14 - EXAMPLE OF USING THE SWAGGER SELF-DOCUMENTING API FOR COMMUNICATING VCG EVENTS TO SLARC. 38

FIGURE 15 - CURRENT SLARC CONFIGURATION FOR INTEGRATING EXTERNAL COMPONENTS TOWARD HANDLING EXTERNAL

EVENTS. .......................................................................................................................................................... 40

FIGURE 16 - THE IT LOAD MARKETPLACE CONNECTOR ARCHITECTURE. ......................................................................... 48

FIGURE 18 - ENERGY AWARE IT LOAD BALANCER - CLIENT ARCHITECTURE. .................................................................... 54

FIGURE 19 - ENERGY AWARE IT LOAD BALANCER - SERVER ARCHITECTURE. .................................................................. 54

FIGURE 20 - IT LOAD MIGRATION SEQUENCE DIAGRAM. ................................................................................................ 56

FIGURE 21 - THE PROCESS VIEW OF THE FEDERATED DC MIGRATION PROCESS. ............................................................ 57

FIGURE 22 - INSTALLATION INSTRUCTIONS FOR SLARC. ............................................................................................. 58

FIGURE 23 – COMPONENTS OF A TYPICAL SLARC INSTALLATION. ................................................................................ 59

FIGURE 24 - SYNCHRONIZING EXISTING VC TAGS FROM A RUNNING VCG INSTANCE. ...................................................... 61



7

List of Tables TABLE 1 - SERVICE OF DCMC SERVER FOR REPORTING ACCEPTED MIGRATION. ............................................................. 22

TABLE 2 - SERVICE OF DCMC SERVER FOR SENDING VC DETAILS. ............................................................................... 22

TABLE 3 - SERVICE OF DCMC SERVER FOR REQUESTING TOKEN. ................................................................................. 23

TABLE 4 - SERVICE OF DCMC SERVER FOR UPDATING VC DETAILS. .............................................................................. 23

TABLE 5 - PARAMETERS REQUIRED FOR INFORMING ABOUT A NEW VC. .......................................................................... 24

TABLE 6 - SERVICE OF DCMC MASTER CLIENT FOR PLACING MIGRATION REQUESTS. ...................................................... 24

TABLE 7 - SERVICE OF DCMC MASTER CLIENT FOR ASSOCIATING MIGRATION WITH THE CATALYST MARKETPLACE

INFORMATION. ................................................................................................................................................. 24

TABLE 8 - SERVICE OF DCMC MASTER CLIENT FOR RETRIEVING MIGRATION ASSOCIATED WITH MARKET ACTION. ............... 25

TABLE 9 - PARAMETERS REQUIRED FOR RETRIEVING MIGRATION ASSOCIATED WITH A CERTAIN MARKET ACTION. ................. 25

TABLE 10 - SERVICE OF DCMC MASTER CLIENT FOR PROVIDING MIGRATION DETAILS. ................................................... 25

TABLE 11 - SERVICE OF DCMC MASTER CLIENT FOR CREATING A VPN. ....................................................................... 26

TABLE 12 - PARAMETERS REQUIRED FOR TRIGGERING A VPN CREATION. ....................................................................... 26

TABLE 13 - SERVICE OF DCMC MASTER CLIENT FOR PROVIDING CTL DETAILS. ............................................................. 26

TABLE 14 - SERVICE OF DCMC MASTER CLIENT FOR TRIGGERING VC CREATION. .......................................................... 27

TABLE 15 - SERVICE OF DCMC MASTER CLIENT FOR RETRIEVING TRANSACTION ASSOCIATED TO VC. ............................... 27

TABLE 16 - PARAMETERS REQUIRED FOR RETRIEVING TRNASCTION ASSOCIATED WITH A CERTAIN VC. ............................... 27

TABLE 17 - SERVICE OF DCMC MASTER CLIENT FOR INFORMING ABOUT VC REGISTRATION. ........................................... 27

TABLE 18 - PARAMETERS REQUIRED FOR INFORMING ABOUT A REGISTERED VC. ............................................................ 28

TABLE 19 - SERVICE OF DCMC LITE CLIENT FOR RECEIVING VPN DETAILS. ................................................................... 28

TABLE 20 - PARAMETERS REQUIRED FOR RECEVING VPN CONNECTION DETAILS. ............................................................ 28

TABLE 21 - MIGRATION DATA MODEL. ........................................................................................................................ 29

TABLE 22 - TOKEN DATA MODEL. .............................................................................................................................. 29

TABLE 23 - VCONTAINERUPDATE DATA MODEL. .......................................................................................................... 29

TABLE 24 – VCONTAINER DATA MODEL. .................................................................................................................... 29

TABLE 25 - MIGRATIONREQUEST DATA MODEL. .......................................................................................................... 30

TABLE 26 - LOADVALUE DATA MODEL. ....................................................................................................................... 31

TABLE 27 - MIGRATION DATA MODEL. ........................................................................................................................ 31

TABLE 28 - MIGRATIONBYACTION DATA MODEL. ......................................................................................................... 31

TABLE 29 - MIGRATIONDETAILS DATA MODEL. ............................................................................................................ 31

TABLE 30 - CONTROLLER DATA MODEL. ..................................................................................................................... 31

TABLE 31 - VCONTAINER DATA MODEL. ...................................................................................................................... 32

TABLE 32 - VCONTAINERUPDATE DATA MODEL. .......................................................................................................... 32

TABLE 33 - VPNDETAILS DATA MODEL. ...................................................................................................................... 32

TABLE 34 - SUPPORTED PROTOCOLS FOR NOTIFYING ON SLA BREAKAGE EVENTS. .......................................................... 36

TABLE 35 - SERVICE ENDPOINT DETAILS FOR RETRIEVING THE LATEST DEFAULT SLA CONFIGURATION. .............................. 38

TABLE 36 - SERVICE ENDPOINT DETAILS FOR SETTING THE DEFAULT SLA CONFIGURATION. .............................................. 39

TABLE 37 - SERVICE ENDPOINT DETAILS FOR RETRIEVING INFORMATION ON ALL THE REGISTERED SLA EVENTS. ................ 39

TABLE 38 - SERVICE ENDPOINT DETAILS FOR CREATING A NEW SLA EVENT. ................................................................... 39

TABLE 39 - SERVICE ENDPOINT DETAILS FOR REGISTERING EXTERNAL EVENTS INTO SLARC. ........................................... 40

TABLE 40 - REQUIRED PARAMETERS FOR PROPERLY INVOKING THE REGISTRATION OF EXTERNAL EVENTS INTO SLARC. ..... 40

TABLE 41 - SERVICE ENDPOINT DETAILS FOR RETRIEVING INFORMATION ABOUT OR MANAGING AN SLA EVENT. .................. 40

TABLE 42 - REQUIRED PARAMETERS FOR PROPERLY INVOKING THE EVENT-HANDLING SERVICES. ..................................... 41

TABLE 43 - SERVICE ENDPOINT DETAILS FOR RETRIEVING INFORMATION ON THE AVAILABLE SLO LEVELS. ......................... 41

TABLE 44 - SERVICE ENDPOINT DETAILS FOR CREATING A NEW SLO LEVEL. ................................................................... 41

TABLE 45 - SERVICE ENDPOINT DETAILS FOR RETRIEVING DETAILS ON OR MANAGE A SLO LEVEL. ..................................... 41

TABLE 46 - REQUIRED PARAMETERS FOR PROPERLY INVOKING THE SLO LEVEL MANAGEMENT SERVICE. ........................... 41

TABLE 47 - SERVICE ENDPOINT DETAILS FOR RETRIEVING INFORMATION ON THE REGISTERED NOTIFICATION CLIENTS. ........ 42

TABLE 48 - SERVICE ENDPOINT DETAILS FOR CREATING A NEW NOTIFICATION CLIENT. ..................................................... 42

TABLE 49 - SERVICE ENDPOINT DETAILS FOR RETRIEVING DETAILS ON OR MANAGING A NOTIFICATION CLIENT. .................... 42

TABLE 50 - REQUIRED PARAMETERS FOR PROPERLY INVOKING THE SERVICE MANAGING NOTIFICATION CLIENTS. ................ 42



8

TABLE 51 - SERVICE ENDPOINT DETAILS FOR RETRIEVING INFORMATION ON THE AVAILABLE SLAS. ................................... 43

TABLE 52 - SERVICE ENDPOINT DETAILS FOR CREATING A NEW SLA. ............................................................................. 43

TABLE 53 - SERVICE ENDPOINT DETAILS FOR RETRIEVING INFORMATION ON AN SLA OF A PARTICULAR ENTITY. ................... 43

TABLE 54 - REQUIRED PARAMETERS FOR PROPERLY INVOKING THE SPECIFIC-ENTITY-SLA RETRIEVAL SERVICE. .................. 43

TABLE 55 - SERVICE ENDPOINT DETAILS FOR RETRIEVING DETAILS ON OR MANAGE A SLA. .............................................. 43

TABLE 56 - REQUIRED PARAMETERS FOR PROPERLY INVOKING THE SLA-MANAGING SERVICE. ......................................... 44

TABLE 57 - SERVICE ENDPOINT DETAILS FOR RETRIEVING THE AVAILABLE SLOS. ............................................................ 44

TABLE 58 - SERVICE ENDPOINT DETAILS FOR CREATING A NEW SLO. ............................................................................. 44

TABLE 59 - SERVICE ENDPOINT DETAILS FOR RETRIEVING INFORMATION ON OR MANAGING A SLO. ................................... 44

TABLE 60 - REQUIRED PARAMETERS FOR PROPERLY INVOKING THE SLO MANAGEMENT SERVICE. .................................... 45

TABLE 61 - DEFAULTS DATA MODEL. .......................................................................................................................... 45

TABLE 62 - SLA EVENT DATA MODEL. ........................................................................................................................ 45

TABLE 63 - THE SLO LEVELS DATA MODEL. ................................................................................................................ 46

TABLE 64 - THE CLIENTTONOTIFY DATA MODEL. ......................................................................................................... 46

TABLE 65 - THE SLO BILLING HISTORY DATA MODEL. .................................................................................................. 46

TABLE 66 - THE SLO DATA MODEL. ........................................................................................................................... 46

TABLE 67 - THE SLA DATA MODEL. ........................................................................................................................... 47

TABLE 68 - SERVICE OF ITMC FOR REGISTERING CANDIDATE MIGRATION ACTIONS.......................................................... 49

TABLE 69 - PARAMETERS REQUIRED FOR REGISTERING CANDIDATE MIGRATION ACTIONS. ................................................ 49

TABLE 70 - SERVICE OF ITMC FOR REGISTERING CANDIDATE MIGRATION ACTIONS FOR SPECIFIC MARKETPLACE TIMEFRAME.

...................................................................................................................................................................... 49

TABLE 71 - PARAMETERS REQUIRED FOR REGISTERING CANDIDATE MIGRATION ACTIONS FOR SPECIFIC MARKETPLACE

TIMEFRAME. .................................................................................................................................................... 50

TABLE 72 - SERVICE OF ITMC FOR RETRIEVING REFERENCE PRICES. ............................................................................ 50

TABLE 73 - PARAMETERS REQUIRED FOR RETRIEVING REFERENCE PRICES. .................................................................... 50

TABLE 74 - SERVICE OF ITMC FOR RETRIEVING REFERENCE PRICES FOR A GIVEN DATE. .................................................. 50

TABLE 75 - PARAMETERS REQUIRED FOR RETRIEVING REFERENCE PRICES FOR A GIVEN DATE. ......................................... 51

TABLE 76 - SERVICE OF ITMC FOR REGISTERING MIGRATION STATUS UPDATES. ............................................................. 51

TABLE 77 - PARAMETERS REQUIRED FOR REGISTERING MIGRATION STATUS UPDATES...................................................... 51

TABLE 78 - MIGRATIONACTION DATA MODEL. ............................................................................................................. 52

TABLE 79 - LOADVALUE DATA MODEL. ....................................................................................................................... 52

TABLE 80 - PRICE DATA MODEL. ................................................................................................................................ 52

TABLE 81 - ACTIONSTATUS DATA MODEL. ................................................................................................................... 52

TABLE 82 - SERVICE ENDPOINT DETAILS FOR RETRIEVING INFORMATION ON ALL KNOWN VC TAGS.................................... 53

TABLE 83 - SERVICE ENDPOINT DETAILS FOR RETRIEVING INFORMATION ON THE VC TAG THAT HAS BEEN REGISTERED AS

MAPPED WITH A GIVEN UUID. ........................................................................................................................... 53

TABLE 84 - PARAMETERS REQUIRED FOR RETRIEVING INFORMATION ON THE VC TAG ASSOCIATED WITH A GIVEN UUID. ..... 53

TABLE 85 - VC TAGS DATA MODEL. ........................................................................................................................... 53

TABLE 86 – NEW INTERFACES OF ITLB CLIENT .......................................................................................................... 55

TABLE 87 - SLARC CONFIGURATION OPTIONS. ........................................................................................................... 59

TABLE 88 - MAPPING OF THE SLARC INSTANCE CONTAINERS AGAINST THE COMPONENT ARCHITECTURE. ......................... 60

TABLE 89 - PORTS EXPOSED BY SLARC. ................................................................................................................... 60



9

List of Acronyms AMPQ Advanced Message Queuing Protocol

API Application Programming Interface

B2B Business to Business

CMP Compute (Node)

CTL Controller

CRUD Create, Read, Update and Delete

DB Database

DC Data Centre

DCMC Federated DC Migration Controller

DCMCL Federated DC Migration Controller Lite Client

DCMCM Federated DC Migration Controller Master Client

DR Demand Response

ETH BC Ethereum Blockchain

IB Information Broker

ID Identity

IDEO Intra DC Energy Optimizer

IP Internet Protocol

IT Information Technology

ITL IT Load

ITLB Energy-aware IT Load Balancer

ITMC IT Load Marketplace Connector

MC Marketplace Connector

QoS Quality of Service

RES Renewable Energy Sources

REST Representational State Transfer

SSH Secure Shell

SLA Service Level Agreement

SLARC SLA (Re)negotiation Controller



10

SLO Service Level Objective

URL Universal Resource Locator

UUID Universally Unique Identifier

UUID4 Universally Unique Identifier version 4

VC Virtual Container

VCG Virtual Container Generator

vCMP Virtual Compute Node

VM Virtual Machine

VPN Virtual Private Network

VPP Virtual Power Plants

WP Work Package



11

Executive Summary

The CATALYST framework applies efficient management of Data Centre flexibility assets under operational,

financial, and sustainability profits. The management of execution of virtual Information Technology (IT) loads

understood as Virtual Containers (VCs) or simply Virtual Machines (VMs) is a potential source of Data Centre

(DC) flexibility, handled and managed by the central Data Centre optimization module. The present document

describes the CATALYST approach to supporting live migration among Data Centres of different

administrative domains, while being transparent to the end user. Specifically, the document provides:

• The Federated DC Migration concept definition within the CATALYST ecosystem, describing the

framework within which migration may happen;

• The Federated DC Migration Process design, which specifies the steps/operations conducted for the

actual migration to happen;

• Technical design information about the components of the CATALYST architecture (D2.3 [1]) involved

in the CATALYST Federated DC Migration, namely the Federated DC Migration Controller, the Service

Level Agreement (SLA) (Re-)negotiation Controller, the IT Load Marketplace Connector, while

references to the already specified Virtual Container Generator (D3.1 [2]) and Energy-aware IT Load

Balancer (D3.2 [3]) are provided.

• Installation instructions for these components and user guidelines under specific scenarios.

This document accompanies the Federated DC Migration Controller, which is available as open source in the

H2020 CATALYST group of the public Gitlab instance [4]. Further, this prototype will be exploited within the

CATALYST framework as the piece of software executing live migration upon request of the Energy-aware IT

Load Balancer, properly integrated with the IT Load Marketplace.



12

1 Introduction The CATALYST project was born at the crossroad of modern IT and energy systems, at an era when cloud

and web technologies are both a bless and a curse. Data Centres (DCs) appear in the middle of the interplay

between the merits of the connected cognitive intelligent world and the cautiousness of the sustainability

drivers related to both environmental, operational and financial constraints. Indeed, the ever more

prominent effects of the climatic change, such as disastrous fires (e.g., the recent Amazon rainforest fires),

increasing events of floods, tornados, and earthquakes worldwide, indicate that there is no opportunity for

squeezing the planet resources anymore. In any case, the increasing operational costs of energy, arising

from increased internet traffic, significantly constrain the profit margin of the DC business.

The turn to green energy sources and rationalized/more efficient management of the DC energy assets is

the key to not only removing the footprint of DCs, but also providing excess energy budgets, thus creating

additional sources of revenue for DC Managers/Owners.

CATALYST envisions green and efficient energy management of DC resources, implementing the innovative

follow-the-energy approach and enabling profits related to reduced energy usage, exploitation of the

available renewable energy and trading of multi-carrier energy budgets in relevant CATALYST-driven

Marketplaces. A potential source of the DC flexibility is the execution of delay-tolerant IT loads, as this is

associated with energy consumption. CATALYST enables live migration of the load of interest among different

administrative domains, while being transparent to the end customer. On the contrary, the terms for the

migration are agreed between the involved DCs on a Business-to-Business (B2B) basis. Indeed, CATALYST

inaugurates a completely new Marketplace vision, developing the IT Load Marketplace, in which federated

DCs may negotiate mutually beneficial migrations of IT loads, effectively exploiting the energy marketplace

principles in the IT business. In this Marketplace variant, DCs may express their interest in offloading to or

accepting IT loads of specified characteristics from remote DCs; such loads have been previously evaluated

locally both as potential energy saving or consumption budgets and as IT loads requiring or releasing IT

resources, while security requirements are appropriately respected.

The present document describes the principles governing the Federated DCs Migration in CATALYST and

provides the methodology and the technical details of achieving live migration of IT loads among different

DCs, without compromising security, service availability, and customer relations.

1.1 Intended Audience The target audience includes mainly DC Managers and Owners, including technical staff. Through this report,

they can get informed about the technical details, understand the advantages and underlying business

model in building B2B relationships via the technical solution described. The CATALYST Federated DC

Migration Process could potentially raise their cautiousness in adopting such a solution.

Moreover, the report can be useful to policy makers, who can find technical advantages of the CATALYST

Federated Migration Process. The sustainability benefits arising from the overall CATALYST solution,

including the migration part, could be promoted, enabling appropriate regulatory changes which can boost

the adoption of the CATALYST framework.



13

As this report is accompanying the CATALYST Federated DC Migration Controller Prototype, it can be useful

to wide developer communities, as well as technology and solution providers wishing to adopt, extend or

alternatively exploit the prototype, respecting its license.

Finally, the report is useful internally, to the members of the development and integration team of the

CATALYST consortium, involving mainly Work Package (WP) 3-WP6 partners, but also to the whole

Consortium for validation and exploitation purposes. Useful feedback could be also received from the

Advisory Board, including both technical and impact creation comments.

1.2 Relations to other activities This document relates to WP2 activities, complying with the identified requirements and implementing parts

of the CATALYST architecture, delivered in D2.1 [5] and D2.3 [1]. Moreover, the Federated DC Migration

Process involves components already implemented within WP3, including the Virtual Container Generator

(VCG) reported in D3.1 [2] and the Energy-aware IT Load Balancer (ITLB) reported in [3]. Moreover, the report

provides useful input for the integration with the IT Load Marketplace, affecting both WP5 and WP6 activities.

Last, but not least, the report can provide feedback to the dissemination and exploitation activities in WP8.

1.3 Document overview The report provides details about the both the concept of the CATALYST federated migration and its technical

realization. The rest of the document is organized as follows:

• Section 2 describes the high-level Federated DC Migration Concept, as well as the high-level

technical approach.

• Section 3 provides implementation details, aligning with the defined CATALYST architecture,

including architectural presentation, specification of interfaces and data model for each component.

• Section 4 provides installation and user’s guidelines for installing/replicating the CATALYST

components involved in the Federated DC Migration Controller prototype.

• Section 5 concludes the report.



14

2 The Federated DC Migration Concept In this section, the CATALYST Federated DC Migration Concept is presented, describing the flow of operations

leading to migration of virtual loads among federated DCs. The high-level concept is described and mapped

to operations/interactions within the CATALYST architecture.

2.1 The Federated DC Migration as Source of CATALYST Flexibility

The CATALYST framework optimizes the management of DC assets, seen as a set of electrical and thermal

flexibility sources against configurable goals of energy consumption reduction, cost benefits or even energy

efficiency and green energy usage maximization. Indeed, CATALYST identifies the migration of delay-tolerant

IT loads (i.e., OpenStack [6] VMs) among federated DCs as a potential source of both electrical and thermal

flexibility in the sense that the IT load execution implies electrical energy consumption and thermal energy

(heat) generation. Thus, IT load migration is subject to DC resources optimization as concluded by the

CATALYST optimization component, i.e., the Intra DC Energy Optimizer (IDEO), which will be presented in

D4.3 “DC Energy Flexibility Management Tool”, planned to be released within the fourth quarter of 2019.

Figure 1 - IT load migration between two DCs.

Indeed, CATALYST handles the migration of loads from DC A to DC B, as depicted in Figure 1. This is

performed in three major steps:

• Identification of loads/resources candidate for migration - This process is induced by the optimized

management of energy generation and consumption units within a single DC, which might suggest

load migration. It is followed by optimization of management of IT resources (Virtual Containers,

servers) to reach energy goals of suggested migration. Loads/resources are finally considered as

candidate for migration, after compliance with the underlying SLA rules. It must be noted that this

step identifies either loads which are candidate for migration or resources which are candidate for

executing migrated loads of remote DCs.

• Trading candidate migrations in the IT Load Marketplace - The DC’s intention to migrate load or offer

resources to execute remote loads is expressed as a (migration) offer or bid, respectively, in the IT

Load Marketplace. The acceptance or rejection of such migration intentions is then subject to the IT

Load Marketplace clearing, which relies on the Market Equilibrium mechanism in the first CATALYST

Marketplace version, as mentioned in D5.1 [7], and has been updated to rely on the CATALYST

optimized Energy-aware IT Load Management, as described in D5.2 [8] and will be presented in the



15

future report D5.3 “Cloudification of the CATALYST market framework”, planned to be released

within the first quarter of 2020.

• Live migration of loads - The actual migration happens when it is granted both at local DC level and

among the involved DCs. This refers to live migration of different OpenStack installations, across

different administrative domains, ensuring security and confidentiality of the communications and

data of involved DCs.

This process is fully compatible and feasible with the CATALYST architecture presented in D2.3. Figure 2

depicts a subset of the CATALYST architecture with the components participating directly or indirectly in the

migration process.

Figure 2 - IT Load Migration as realized in the CATALYST architecture.

In the following subsections these three steps leading to load migration are further analysed under the

CATALYST architecture perspective, describing the role and interaction of involved CATALYST components.

2.1.1 Identification of loads/resources candidate for migration IT load migration involves the management of assets of multiple DCs and potentially affects the Quality of

Service (QoS) for DC’s customers. CATALYST follows a carefully designed process before each single

migration action actually takes place, considering the optimized management of IT loads, DCs’ agreement

and end-user satisfaction. In brief, the identification of loads or resources which are approved for migration

is derived from the following process.

1) The Intra DC Energy Optimizer (IDEO) extracts an optimization plan, which may include an

optimization action referring to increasing (decreasing) electrical energy consumption by a certain

rate through migrating loads (accepting remote loads for execution).



16

2) The DC Operator selects an optimization plan, which includes an action relevant to load migration.

3) The Energy-aware IT Load Balancer (ITLB) Client retrieves the selected optimization plan and

proposes a configuration of optimal placement of tolerant loads of a single DC across servers and

DCs.

4) (a) The ITLB Client asks the SLA (Re)negotiation Controller (SLARC) about the SLA status of the VCs

of the intended migration. (b) Then SLARC queries the VCG Server about the association of the VCs

of interest to VC tags (which are understood by SLARC), (c) which are provided by the VCG Client,

and, based on that, provides the SLA status to the ITLB Client.

5) If the SLA status permits it, the Federated DC Migration Controller (DCMC) is informed about the

intended migration.

2.1.2 Trading candidate migrations in the IT Load Marketplace A migration action among the source and the destination DCs is decided within the IT Load Marketplace.

DCs participate in this marketplace by placing migration offers (i.e., loads) or migration bids (i.e., resources),

but their acceptance relates to the inherent IT Load Marketplace operations related to market clearing.

Details on the market clearing mechanism of the first version of the IT Load Marketplace can be found in

D5.1, while updates are reported in D5.2 report. Within the CATALYST architecture, this step is realized

through the following process.

6) DCMC triggers the process of trading load migrations with remote DCs.

7) The IT Load Marketplace Connector (ITMC) is notified and takes over the communication with the

Marketplace components. First, it creates and places a relevant action in the IT Load Marketplace;

an offer for actions suggesting migrating certain IT loads in remote DCs or a bid for actions asking

for IT loads to be executed on the current DC. ITMC is later notified about the result of the market

clearance, essentially informing about the acceptance or rejection of the planned migration.

8) DCMC and VCG are informed accordingly.

2.1.3 Live migration of loads The third and final step of the migration flow includes the actual migration task and is realized as follows

within the CATALYST architecture.

9) Upon acceptance of their migration offer or bid, the DCMC clients together with the server perform

the live migration between the involved DCs.

10) Upon completion of this task, (a) ITLB, (b) VCG, and (c) ITMC are informed about the success or failure

of the migration. The latter informs also the Marketplace components, which collect invoicing

information.

11) (a)VCG notifies SLARC about the state update of the VC and (b) persists the migration as a

transaction in the blockchain, associating it with the VC tag of the load in question. Future

transactions regarding this specific VC tag (thus load) will ensure the traceability of that load

migrations across the CATALYST ecosystem.

With this process, CATALYST ensures that migration is attempted only for delay-tolerant loads, for which DC

Operators’ opinion and SLA management regarding the migration in question are duly considered. It must

be noted that live migration is possible among DCs belonging to the same or different administrative

domains.



17

Section 2.2 describes in detail the steps leading to VCs’ live migration mentioned in Step 9) above and later

referred to as “Federated DC Migration Process”.

2.2 Realizing Federated DC Migration DCMC realizes a client-server architecture and thus composed of three counterparts:

a) DCMC server, appearing as DCMC in the next figures and which is responsible of managing

operations of the load migration among DCs, such as keeping DC identification details, setting up

secure communication channels among DCs, etc.

b) DCMC master client, appearing as DCMCiM (where i is a number indicating the hosting DC) in the next

figures and which take preparatory actions on the source and destination DCs for the migration to

be possible and perform the live migration.

c) DCMC lite client, appearing as DCMCiL, which allows monitoring and communication with the

migrated load.

2.2.1 Federated DC Migration Process Design The Federated DC Migration Process defines the flow of operations and interaction of the DCMC

counterparts, which lead to the load migration. For the sake of the analysis of this process, let assume that

a migration action has been accepted in the IT Load Marketplace, referring to load of DC1 being accepted

to migrate to DC2. The following analysis also assumes that migration takes place among OpenStack

installations of the involved DCs.

The process starts when DCMC1M and DCMC2M are notified by their IT Load Marketplace Connectors

(ITMC1 for DC1 and ITMC2 for DC2). Both components inform that they have been granted for migration;

DCMC1M at the source DC (DC1) and DCMC2M at the destination DC (DC2) related to transaction T in the

CATALYST Marketplace.

Figure 3 - Preparing for migrating load from DC1 to DC2.

The first priority in order to realize migration among DCs is the establishment of a secure communication

channel. So, DCMC1M establishes a Virtual Private Network (VPN) between DC1 and DC2 (Step 1 in Figure



18

3). As the remote load will be executed in a new, specially created VC on DC2, DCMC notifies DCMC2M to

create it (2). DCMC2M responds, by first ensuring communication of this new VC with the DCMC, in order to

allow communication of migration details. Thus, DCMC2M asks DCMC for a token for the new VC to connect

to DCMC (3). A new VC is then created (4), in order to accommodate the remote load. This VC must not be

accessible internally by DC2 for reasons of data security and privacy. So, no Secure Shell (SSH) key file is

available for this VC, which would allow remote connection to it by the host compute (CMP) node or any other

computer. DCMC2L of the new VC contacts DCMC, using its token, to inform about the reason of its existence,

that it has been created for transaction T (5). Now that DCMC knows that the ground is ready for the migration

at the destination DC, it provides DCMC2L with the VPN details and the Internet Protocol (IP) address of

DCMC1M within the VPN (6). Figure 3 depicts the process up to this point.

Figure 4 - Live migration of load from DC1 to DC2.

Now everything is ready for the secure connection between the two DCs, so the DCMC2L connects to the

VPN and contacts DCMC1M, asking for details to reach the Keystone Identity Service1 [9] of DC1, usually

residing on a Controller (CTL)2 node (7). With the relevant details at hand, DCMC2L registers the new VC to

the CTL node as a virtual compute node (vCMP) (8). DCMC2L notifies DCMC1M that vCMP has been

registered to the CTL (9). DCMC1M notifies the Nova service3 [10] (usually residing in the CTL) to perform

migration from the original VC to the newly registered, remote vCMP (10). DC1 CTL initiates the live migration

of the load, which may reside in one or more CMP nodes. The status of the migration at this point is

“pending”. Live migration is conducted between CMP and vCMP (11). If this action is successful, the

migration status is marked as “success”, while a “pending” migration after the end of the delivery period (in

CATALYST marketplace terms – see D5.1 for details) denotes a failed migration (12). Figure 4 presents DC1

and DC2 after a successful migration.

1 Keystone is an OpenStack service that provides API client authentication, service discovery, and distributed multi-tenant authorization.

2 This is an OpenStack Controller, common in most OpenStack configurations: https://docs.openstack.org/arch-design/use-cases/use-case-general-compute.html

3 Nova is the OpenStack project that provides a way to provision compute instances (virtual servers).

https://docs.openstack.org/arch-design/use-cases/use-case-general-compute.html

https://docs.openstack.org/arch-design/use-cases/use-case-general-compute.html



19

Summarizing, the CATALYST migration concept has been presented as a flow of operations within the

CATALYST system. These operations draw a comprehensive conceptual picture of the technological approach

of CATALYST towards the realization of live migration among different OpenStack installations.



20

3 Implementation In this section, the technical design of the CATALYST components implementing the Federated DC Migration

Concept of CATALYST is presented.

3.1 Federated DC Migration Controller

3.1.1 Architecture DCMC is realized as a distributed component, composed of:

• The DCMC Server (DCMC)

• The DCMC Master Client (DCMCM)

• The DCMC Lite Client (DCMCL)

The architecture of the three DCMC types is depicted in Figure 5.

(a) The DCMC Server. (b) The DCMC Master Client. (c) The DCMC Lite Client.

Figure 5 - The DCMC architecture.

3.1.1.1 DCMC Server The DCMC Server lies in the CATALYST DCs’ Federation part of the CATALYST architecture, and keeps records

of DCs in the federation, sets up secure communication channels among them and triggers the initiation of

the migration process. In more detail, DCMC undertakes the tasks of:

• Maintaining identification information of DCs participating in the federation;

• Keeping track of the client instances of the federated DCs;

• Managing the creation of a new VC at the destination DC for the remote load execution.

Such functionality is supported by three components of the DCMC Server:



21

• The Representational State Transfer (REST)-ful Application Programming Interface (API), allowing

DCMC client instances to interact with it;

• The Migration Manager, which collects notifications of approval of migration and triggers the

bootstrapping of the migration process. Such notifications are realized as notifications by the DCMCM

clients of both the source and the destination DCs that their migration offer and bid, respectively,

has been accepted, referring to the same migration instance.

• The Secure Access Manager, which is responsible for identity and access management of

components interacting with the DCMC.

• The Federation DB, where it keeps identification/authorization details of DCs registered in the

federation, as well as information about placed migration actions.

The DCMC Server interacts only with the DCMCM and DCMCL instances residing on the federated DCs.

3.1.1.2 DCMC Master Client The DCMC Master Client resides at every DC of the federation and performs the preparatory tasks for the

actual migration. Normally, this component will be found in the Controller of the underlying OpenStack

installation, but it could be running in other nodes, as well, depending on the OpenStack configuration.

DCMCM is mainly responsible for supporting load migration, from its conception as an idea drawn by the

ITLB, until its final delivery. The main functionalities of this component can be summarized as follows.

• Monitoring the state of virtual loads and informing VCG accordingly, so it can keep track of the loads’

state;

• Managing the host DC’s participation in the IT Load Marketplace, by triggering market actions’

placement, as a result of relevant notifications;

• Upon acceptance of load migration actions in the IT Load Marketplace, notifying the DCMC Server;

• Setting up secure communication channels among the source and destination DCs;

• Creating and sharing connection details with client instances over the secure communication

channel.

• Ensuring communication of the newly created VC with the DCMC server;

• Communicating the migration result to the interested (CATALYST) parties.

Such tasks are supported by the following subcomponents:

• The RESTful APIs, namely the Northbound API towards DCMC, DCMCL, and the Southbound API

towards ITLB, and ITMC, as appearing in Figure 5(b), ensuring interaction with those components.

• The Migration Database (DB), where information about intended, ongoing or past migrations is kept.

• The Migration Manager, which undertakes the management of actions related to handling

notifications by other components and triggering migration related tasks to the relevant components.

• The Secure Communication Manager, which ensures the setup of the VPN and connection to it.

• The VC State Monitor, which catches events of VC state changes and notifies VCG accordingly.

DCMCM interacts with:

• The DCMC Server;

• DCMCM of the other end;

• DCMCL (of the destination DC);

• The VCG Client;

• The ITLB Client;

• The ITMC.



22

3.1.1.3 DCMC Lite Client The DCMC Lite Client is a reduced version of the DCMC Master Client. It is instantiated inside the node which

hosts the load originally in DC1 and in the (new) VC in which the migrated load will be executed. So, its main

responsibility is to transfer and execute the load remotely.

From an architectural point of view, this functionality is delivered via three subcomponents;

• A RESTful API towards DCMC;

• The Secure Communication Manager, which allows the Lite Client to interact with the original DC

Controller via the VPN;

• The Migration Manager, which receives the load from the source DC and manages its execution.

DCMCL interacts with:

• The DCMCM of both the source and destination DCs;

• The Keystone service of the source DC;

• The DCMC Server.

3.1.2 Interfaces Specification

3.1.2.1 DCMC Server

3.1.2.1.1 Report accepted migration

This service is meant to be used by DCMC Master Clients of both the source and destination DCs to report

accepted migration bid or offer (in the IT Load Marketplace) to the DCMC Server. Specification details are

provided in Table 1.

Table 1 - Service of DCMC Server for reporting accepted migration.

URL /migration/

Method POST

Headers Content-Type: application/json

Authorization: Bearer DCMC_S_TOKEN

Request body Migration (see Table 21)

Response body N/A

Response codes 201 – Reported accepted migration. Migration object was created.

400 – Provided data is invalid or malformed

401 – Unauthorized users are not allowed to create resources

403 – Operation forbidden for this type of user

405 – This type of method is not allowed for this endpoint

500 – Internal server error

3.1.2.1.2 Send VC details

This service is meant to be used by the DCMC Master Client of the source DC to inform the DCMC Server

about the details of the load candidate for migration. Specification details are provided in Table 2.

Table 2 - Service of DCMC Server for sending VC details.

URL /vcontainer/



23

Method POST



Request body VContainer (see Table 24)

Response body N/A

Response codes 201 – VC Creation was triggered. VC Object was created.






3.1.2.1.3 Request token

This service is meant to be used by the DCMC Master Client of the destination DC to request token on behalf

of the new VC (which will host the migrated load) for accessing the VPN set by the DCMC Server. Specification

details are provided in Table 3.

Table 3 - Service of DCMC Server for requesting token.

URL /token/

Method POST


Request body Token (see Table 22)

Response body N/A

Response codes 201 – Access Token was created

400 –Provided data is invalid or malformed

404 – The Keycloak Authorization URL was not found. No token was created.



3.1.2.1.4 Update VC details

This service is meant to be used by the DCMC Lite Client to inform the DCMC Server that it has been

created for a certain transaction and is ready to accept migrated load. Specification details are provided in

Table 4 and Table 5.

Table 4 - Service of DCMC Server for updating VC details.

URL /transaction/<transaction_id>/vcontainer/status/created/

Method POST



Request body VContainerUpdate (see Table 23)

Response body N/A

Response codes 201 – Updated VC details. Transaction object was created.








24

Table 5 - Parameters required for informing about a new VC.

Parameter Type Comments Example value

transaction_id Integer The id of the transaction, as this is kept in the

CATALYST IT Load Marketplace and

communicated to DCMCL by the DCMC1M.

1

3.1.2.2 DCMC Master Client

3.1.2.2.1 Place a migration request

This service is meant to be used by the ITLB in order to place new migration requests. Specification details

are provided in Table 6.

Table 6 - Service of DCMC Master Client for placing migration requests.

URL /migration/request/

Method POST


Authorization: Bearer DCMC_M_TOKEN

Request body MigrationRequest (see Table 25 and Table 26)

Response body N/A

Response codes 201 – Migration request was placed. Migration object was created


401 – Unauthorized users are not allowed to view this resource




3.1.2.2.2 Associate migration with the CATALYST Marketplace information

This service is meant to be used by the ITMC in order to associate a certain migration record with the relevant

market action it has placed in the IT Load Marketplace. Specification details are provided in Table 7.

Table 7 - Service of DCMC Master Client for associating migration with the CATALYST Marketplace information.

URL /migration/marketplace/update/

Method POST



Request body Migration (see Table 27)

Response body N/A

Response codes 201 – Migration was associated with the CATALYST Marketplace Information








25

3.1.2.2.3 Retrieve migration associated with market action

This service is meant to be used by the ITMC in order to retrieve the details of migration which is associated

with a certain market action. Specification details are provided in Table 8 and Table 9.

Table 8 - Service of DCMC Master Client for retrieving migration associated with market action.

URL /migration/marketaction/<action_id>/

Method GET

Headers Authorization: Bearer DCMC_M_TOKEN

Request body N/A

Response body MigrationByAction (see Table 28)

Response codes 200 – Everything went well



404 – Request migration not found



Table 9 - Parameters required for retrieving migration associated with a certain market action.


action_id Integer The id of the market action, corresponding to

the migration of interest.

1

3.1.2.2.4 Provide migration details

This service is meant to be used by the ITMC in order to provide information about accepted migrations (in

the IT Load Marketplace). Specification details are provided in Table 10.

Table 10 - Service of DCMC Master Client for providing migration details.

URL /migration/details/

Method POST



Request body MigrationDetails (see Table 29)

Response body N/A

Response codes 201 – Migration details were provided. Details object was created.




404 – No load or migration objects were found by this market action



3.1.2.2.5 Request VPN creation



26

This service is meant to be used by DCMC Server in order to ask the DCMC Master Client to create a VPN for

the communication among the DCs which will participate in the imminent migration. Specification details are

provided in Table 11 and Table 12.

Table 11 - Service of DCMC Master Client for creating a VPN.

URL transaction/{transaction_id}/vpn/create/

Method POST



Request body -

Response body -





404 – No details were found for the provided transaction



Table 12 - Parameters required for triggering a VPN creation.


transaction_id Integer The id of the transaction in the CATALYST IT

Load Marketplace.

1

3.1.2.2.6 Provide CTL details

This service is meant to be used by the DCMC Lite Client to retrieve the details of the node hosting the

OpenStack Keystone service (usually the Controller). Specification details are provided in Table 13.

Table 13 - Service of DCMC Master Client for providing CTL details.

URL /controller/

Method GET


Request body N/A

Response body Controller (see Table 30)




404 – Requested resource not found



3.1.2.2.7 Trigger VC creation

This service is meant to be used by the DCMC Server to notify the DCMC Master Client of the destination DC

to create a new VM for a certain transaction. Specification details are provided in Table 14.



27

Table 14 - Service of DCMC Master Client for triggering VC creation.

URL /vcontainer/

Method POST



Request body VContainer (see Table 31)

Response body N/A

Response codes 201 – VC Creation was triggered. VC Object was created






3.1.2.2.8 Retrieve transaction associated to VC

This service is meant to be used by the DCMC Lite Client to retrieve the transaction id, which is associated

with a certain VM, normally created to execute the load which will be migrated. Specification details are


Table 15 - Service of DCMC Master Client for retrieving transaction associated to VC.

URL /transaction/vcontainer/{uuid}/

Method GET


Request body N/A

Response body VContainerUpdate (see Table 32)




404 – Requested resource not found



Table 16 - Parameters required for retrieving trnasction associated with a certain VC.


uuid String Identification information of the VM, for which

the associated transaction is requested.

vdfkbjafhvlafjven

dflvkdsfnvjndsflv

kshdl

3.1.2.2.9 Inform about VC registration

This service is meant to be used by the DCMC Lite Client in order to inform the DCMC Master Client of the

source DC that a VC has been registered to the CTL, so it is ready for migration. Specification details are


Table 17 - Service of DCMC Master Client for informing about VC registration.

URL /transaction/{transaction_id}/vcontainer/status/registered/

Method POST



28



Request body VContainerUpdate (see Table 32)

Response body N/A


400 –Provided data is invalid or malformed



404 – The resource under update was not found



Table 18 - Parameters required for informing about a registered VC.


transaction_id Integer The id of the transaction in the CATALYST

IT Load Marketplace.

1

3.1.2.3 DCMC Lite Client

3.1.2.3.1 Receive VPN details

Through this service connection details for the VPN are provided to the DCMC Lite Client for contacting

components of the source DC. Specification details are provided in Table 19 and Table 20.

Table 19 - Service of DCMC Lite Client for receiving VPN details.

URL /transaction/{transaction_id}/connection/

Method POST


Request body VpnDetails (see Table 33)

Response body N/A


400 – The provided data is invalid or malformed


Table 20 - Parameters required for receving VPN connection details.


transaction_id Integer The id of the transaction in the CATALYST IT

Load Marketplace.

1



29

3.1.3 Data Model

3.1.3.1 DCMC Server

3.1.3.1.1 Data Model

The data model used for consuming the DCMC Server services is presented in Table 21 to Table 24.

Table 21 - Migration data model.

Property Type Description

dc_name String The username of the issuing DC, as this is understood by the DCMC Server

role String

The role of the issuing DC in the migration (can be either “source” or

“destination”)

delivery_start Datetime The time on which the migration is planned to start

delivery_end Datetime The time on which the remote execution is planned to end

transaction Integer

The id of the transaction, as this is kept in the CATALYST IT Load

Marketplace

Table 22 - Token data model.


username String

The username of the issuing DC for gaining authorization in the DCMC

Server

password String

The password of the issuing DC for gaining authorization in the DCMC

Server

Table 23 - VContainerUpdate data model.


transaction Integer


Marketplace

vcontainer String Identification information of the issuing VM (UUID)

Table 24 – VContainer data model.


cpu Integer The number of CPU cores of the VC described

cpu_uom String The unit of measurement for the “cpu” value (usually “cores”)

ram Integer The amount of RAM of the VC described

ram_uom String The unit of measurement for the “ram” value

disk Integer The amount of disk of the VC described

disk_uom String The unit of measurement for the “disk” value

start Datetime The time from which the VC is planned to be active

end Datetime The time until which the VC is planned to be active

uuid String Identification information of the issuing VM (uuid)



30

3.1.3.2 DCMC Master Client

3.1.3.2.1 Database Schema

The database schema of the Migration DB of the DCMC Master Client is depicted in Figure 6.

Figure 6 - The DCMC Master Client database schema.

3.1.3.2.2 Data Model

The data model for consuming the services of the DCMC Master Client is presented in Table 25 to Table 32.

Table 25 - MigrationRequest data model.


date Datetime The time on which the request is created

vc_tag String The VC tag, as this has been generated by the VCG

starttime Datetime The time on which the requested migration is planned to start

endtime Datetime The time on which the remote execution is planned to end

load_values LoadValue[] Information about the load characteristics

price Decimal The price suggested for the migration action

action_type String

The type of market action, which will be placed on the CATAYST IT Load

Marketplace (can be either “bid” or “offer”)



31

Table 26 - LoadValue data model.


parameter String The name of the load characteristic (can be “cpu”, “ram” or “disk”)

value Decimal The value quantifying the load characteristic

uom String The unit of measurement of the value, depending on the “parameter”.

Table 27 - Migration data model.


marketaction Integer

The id of the market action (in the IT Load Marketplace), corresponding to

the migration

migration Integer The id of the migration

status String

The status of the migration. It can take one of the values; "pending",

"posted", "rejected", "started", "success", "failed".

Table 28 - MigrationByAction data model.




the migration

migration Integer The id of the migration

Table 29 - MigrationDetails data model.




the migration

source String The id of the source DC of the migration

destination String The id of the destination DC of the migration

delivery_start Datetime The time on which the migration is planned to start

delivery_end Datetime The time on which the remote execution is planned to end

transaction Integer


Marketplace

Table 30 - Controller data model.


name String

The name of the OpenStack node hosting the Keystone service in the

source DC

ip String The ip of this node



32

Table 31 - VContainer data model.


cpu Integer The number of CPU cores of the VC described

cpu_uom String The unit of measurement for the “cpu” value (usually “cores”)

ram Integer The amount of RAM of the VC described

ram_uom String The unit of measurement for the “ram” value

disk Integer The amount of disk of the VC described

disk_uom String The unit of measurement for the “disk” value

start Datetime The time from which the VC is planned to be active

end Datetime The time until which the VC is planned to be active

uuid String Identification information of the issuing VM (uuid)

Table 32 - VContainerUpdate data model.


transaction Integer


Marketplace

vcontainer String Identification information of the issuing VM (uuid)

status String The status of the VC (can be “created” or “registered”)

3.1.3.3 DCMC Lite Client The data model for consuming the services of the DCMC Lite Client is depicted in Table 33.

Table 33 - VpnDetails data model.


vpn_ip String The IP of the VPN, in which the DCMC Lite Client will be connected

dcmc1m_ip String The VPN IP of the DCMC Master Client of the source DC

3.2 SLA (Re-)negotiation The CATALYST SLA (Re-)negotiation (controller) (SLARC) is the component responsible for monitoring the SLA

compliance of the CATALYST VCs, based on the events registered in the VCG.

Conceptually, SLARC bases its operation on the notion of Service Level Objectives (SLOs) [11]. Particularly,

every SLA should be associated with one or more SLOs pertaining to the same entity (VC Tag), which

represent “objectives” which are desired to be covered via the underlying SLA. Indeed, every SLO defines a

leveled acceptable behaviour against a target objective for given periods, i.e. the billing periods of virtual

load usage. Indicatively, an SLA related to VC Tag, e.g., V, could define two SLOs as follows:

• SLO A considering a VC downtime of a maximum of 10 minutes per month as acceptable;

• SLO B considering the hosting of a VC in non-green-powered services for a maximum of 1 hour per

day as acceptable.

Each SLO should be characterized by at least two SLO levels that define how the pricing of the SLA due to

an SLO breakage level should be affected. Continuing the example, we could consider:



33

• SLO A levels

a. Level 1: Downtime of 0 – 5 minutes per month results in no price reduction;

b. Level 2: Downtime of 5 – 10 minutes per month results in 40% price reduction;

c. Level 3: Downtime of 10 or more minutes per month results in 100% price reduction.

• SLO B levels

a. Level 4: Hosting in brown servers for less than 1 hour per day results in no price reduction;

b. Level 5: Hosting in brown servers for 1 or more hours per day results in 20% price reduction.

Every time a relevant (e.g., downtime) event gets communicated to the SLARC services for a VC Tag, an

asynchronous task gets fired by the Tasks Manager instances (see section 3.2.1), measuring the event

duration with a one-second precision. When a new event indicating that the previous (downtime) event has

come to an end gets received, the event gets marked as ‘completed’ and its duration gets added to the

current SLO value. Based on this value, the currently active SLO level gets calculated and the SLO status

gets determined. Figure 7 overviews the conceptual hierarchical structure governing the SLARC operation

and SLO status calculation.

Figure 7 - Conceptual base SLARC data model architecture.

It is worth noting that every minute, a check for expired SLAs is performed (events corresponding to expired

SLAs are ignored from SLARC). Similarly, every two minutes, a check for SLA breakage is done. The frequency

of the above scheduled operations is configurable from the SLARC settings. In both cases, notifications to

the responsible entities and actors (as registered in SLARC – see paragraph 3.2.2 for details) are

automatically sent whenever deemed necessary. It should be noted that notifications are sent prior to SLA

breakage, based on a percentage of the SLO value against the maximum allowable value of the last SLO

level. With respect to the previous example concerning downtime, if a user selected to be notified at 80% of

the SLO level, she would get a notification when the SLO value would surpass the 8 minutes during the billing

period (per month).

The following sequence diagrams, shown in Figure 8 and Figure 9, overview in a simplified manner the

general SLARC control flow when input indicating an event start/end is being received. The perspective of

the presentation is CATALYST oriented in the sense that the event-generating entity is considered to be VCG.

The SLO considered is related to VC downtime.

Whenever an event with no end time gets received (that is an event that could be considered as a new one),

SLARC checks whether such an event is already handled (e.g., due to a previous relevant message) and if

not, a new event value-calculating task gets spawned, updating the event and SLO values every one second.

This is depicted in Figure 8.



34

Figure 8 - Handling new external events.

The execution of the event-valuating task ends only when another external event indicates that the event

came to an end. This is depicted in Figure 9.

Figure 9 - Handling the finalization of external events.

3.2.1 Architecture Architecturally, SLARC has been designed using a microservices-oriented approach, essentially comprising

six (6) base components, each of which may have multiple instantiations within the same SLARC installation.

Figure 10 depicts the overall SLARC high-level architecture.



35

The core components of the architecture are mainly the API and the Tasks Manager instances, responsible

for handling the incoming requests (implementing the Southbound API towards ITLB and Northbound API

towards VCG) and the management of event-valuating and scheduled (for notification purposes) tasks. These

components make use of one database each, used for information (for the API) and tasks state (for the

Tasks Manager instances) handling. Further, assisting components include a Cache server and a Task Queue

Manager allowing the API and the Tasks Manager to effectively exchange data and control messages to

interoperate.

It is worth noting that the API service is masked by a load balancing service (transparent to external

applications and components), so that multiple API instances may be spawned in case of heavy SLARC

traffic. Similarly, multiple instances of the Task Manager service may be deployed in case the available task

managing threads fall short to serve the event-valuating processes on time. The default value of Task

Manager instances is set to three (3).

Figure 10 - The SLA (Re-)negotiation Controller architecture.

Technology-wise, the API is implemented using Django REST framework [12], whereas the Tasks Manager

instances are implemented using the Celery software stack [13]. The state DB instances (used by the API

and the Tasks Manager instances) are instances of PostgreSQL [14]. The cache technology is based on the

Redis in-memory data store [15] whereas the Task Queue Manager uses the RabbitMQ [16] messaging

server implementing the Advanced Message Queuing Protocol (AMQP) standard [17]. The load balancing

service is based on Nginx [18]. Finally, as documented in sub-section 4.1.3.1, packetization of SLARC is

performed using container technologies, Docker [19] in particular.

3.2.2 Interfaces Specification SLARC features two different methods for interfacing with external components, that is a synchronous and

an asynchronous one. The synchronous approach relates to the SLARC self-exposed API documented in this

sub-section. The asynchronous approach pertains to the SLA notifying other entities (e.g., the ITLB) about

possible breakages in SLAs. The relevant notifications are generated by the Tasks manager. Three basic



36

protocols have been considered for enabling the communication of the notifications to the interested parties

including REST (HTTP) and Publish/Subscribe (AMQP, KAFKA [20]) modalities, as presented in Table 34.

Table 34 - Supported protocols for notifying on SLA breakage events.

Protocol Description Comments

HTTP/HTTPS Perform a POST request to an HTTP(s) service

KAFKA Publish to an external Kafka service

AMQP Publish to the internal Tasks Queue Manager

service, exposing a RabbitMQ interface.

The queue is always “catalyst”, the

exchange is always

“catalyst.sla.notifications”.

It is worth noting that (as will be later detailed) SLARC defines a specific interface for registering notification

clients per SLA. This allows for the definition of multiple notification clients per SLA; a single SLA could trigger

SLA breakage notifications to a multitude of (possibly heterogeneous) services. In the following figures,

examples of SLARC notifying an external Kafka service (Figure 11) and the internal Task Queue Manager

(Figure 12) are provided. By default, the SLA breakage conditions are checked every two minutes; however,

this behaviour is customizable. More details on this matter are provided in sub-section 4.1.3.1, Table 87.

Figure 11 - SLARC notifying for SLA breakage an external Kafka interface.



37

Figure 12 - SLARC notifying for SLA breakage the internal Tasks Queue Manager.

Regarding the synchronous interaction with SLARC, a RESTful interface has been defined and its

specification is available in the form of a self-documenting API following the OpenAPI specification [21]. The

user interface is powered by Swagger [22], as depicted in Figure 13 and Figure 14.

Figure 13 - SLARC high-level API specification.



38

Figure 14 - Example of using the Swagger self-documenting API for communicating VCG events to SLARC.

In the following, the full list of the service API endpoints exposed by SLARC is provided. In all cases, Create,

Read, Update and Delete (CRUD) operations are available, supported by several accompanying interfaces

facilitating data discoverability. It should be noted that all objects are deeply served, namely they

encapsulate all relevant information (e.g., an SLA contains information related its relevant SLOs, the SLO

contains information on its composing levels etc.).

3.2.2.1 Default configuration for new SLAs This set of interfaces allows the SLARC users to retrieve or create default settings in order to handle VC Tags

that have not been a-priori registered in SLARC. In other words, whenever an event corresponding to an

unknown VC Tag arrives in SLARC, a “default” SLA stack (SLA, SLO and levels) gets generated based on the

values of the latest Defaults entry in the database (see Table 61 for details and discussion). Specification

details of the service endpoints are provided in Table 35 and Table 36.

Table 35 - Service endpoint details for retrieving the latest default SLA configuration.

URL /api/defaults/

Method GET

Headers Accept: application/json

Request body N/A

Response body Defaults (see Table 61)

Response codes 200 - Everything went ok



39

Table 36 - Service endpoint details for setting the default SLA configuration.

URL /api/defaults/

Method POST


Request body Defaults (see Table 61)

Response body N/A

Response codes 201 – Created

3.2.2.2 SLA Events handling This set of interfaces allows the SLARC users to retrieve information or handle events that pertain to specific

SLOs (hence SLAs). Based on the handling of these events, the respective SLO values get calculated as the

sum of the values of the events that are relevant to the billing period of the SLO. Specification details for the

service endpoints are provided in Table 37 and Table 38.

Table 37 - Service endpoint details for retrieving information on all the registered SLA events.

URL /api/events/

Method GET


Request body N/A

Response body List<Event> (see Table 62)

Response codes 200 – Everything went ok

Table 38 - Service endpoint details for creating a new SLA event.

URL /api/events/

Method POST


Request body Event (see Table 62)

Response body N/A

Response codes 201 - Created

The following endpoint, specified in Table 39 and Table 40, is used as an integration point with external

event-generating components such as the VCG. These components can send their input to this endpoint,

granted that a relevant event handler for that component has been registered into the SLARC system. The

relevant registration process includes writing the entry in the api.views module (in the

ExternalComponents class create handler) and creating the relevant client (and handlers) under the

api.clients module. Figure 15 overviews the current SLARC configuration for handling such events. In

the red boxes the place where the new component event-receiving code should be placed is depicted,

whereas the blue box indicates the recommended structure that a new external events’ handling client code

should follow.



40

Table 39 - Service endpoint details for registering external events into SLARC.

URL /api/events/external/{component}

Method POST


Request body Generic

Response body N/A


Table 40 - Required parameters for properly invoking the registration of external events into SLARC.


component string The component identifier vcg

Figure 15 - Current SLARC configuration for integrating external components toward handling external events.

The following endpoint, specified in Table 41 and Table 42, may be used for retrieving details about a specific

event or managing it.

Table 41 - Service endpoint details for retrieving information about or managing an SLA event.

URL /api/events/{id}

Method GET/PUT/PATCH/DELETE


Request body N/A

Response body Event (see Table 62)


204 – No content (only for DELETE)



41

Table 42 - Required parameters for properly invoking the event-handling services.


id string The id of the event be781477-0130-4b76-b289-022d478c80c0

3.2.2.3 SLO levels handling The following set of CRUD APIs (specified in Table 43 and Table 44) pertains to the handling of the levels

that define the various SLO levels.

Table 43 - Service endpoint details for retrieving information on the available SLO levels.

URL /api/levels/

Method GET


Request body N/A

Response body List<Level> (see Table 63)


Table 44 - Service endpoint details for creating a new SLO level.

URL /api/levels/

Method POST


Request body Level (see Table 63)

Response body N/A


As in the previous API service cases, the following endpoint, specified in Table 45 and Table 46, may be used

to retrieve details or manage an SLO level.

Table 45 - Service endpoint details for retrieving details on or manage a SLO level.

URL /api/levels/{id}



Request body N/A

Response body Level (see Table 63)



Table 46 - Required parameters for properly invoking the SLO level management service.


id string The id of the SLO Level 330af0cb-db65-4232-a41c-9c12311b6ad2



42

3.2.2.4 Client notification services The following set of CRUD APIs allow SLARC users to subscribe for SLA-related notifications. The notification

rules are bound to specific SLAs hence entities (VC Tags). However, as already hinted, the determination of

whether a notification should be sent or not is performed on the basis of SLOs. API specifications are

provided in Table 47 to Table 50.

Table 47 - Service endpoint details for retrieving information on the registered notification clients.

URL /api/notify/clients/

Method GET


Request body N/A

Response body List<ClientToNotify> (see Table 64)


Table 48 - Service endpoint details for creating a new notification client.

URL /api/notify/clients/

Method POST


Request body ClientToNotify (see Table 64)

Response body N/A


Table 49 - Service endpoint details for retrieving details on or managing a notification client.

URL /api/notify/clients/{id}



Request body N/A

Response body ClientToNotify



Table 50 - Required parameters for properly invoking the service managing notification clients.


id integer A unique integer value identifying the requested

notification client

17

3.2.2.5 SLAs handling This set of CRUD APIs, specified in Table 51 and Table 52, allows users to retrieve, create, and manage SLAs.



43

Table 51 - Service endpoint details for retrieving information on the available SLAs.

URL /api/slas/

Method GET


Request body N/A

Response body List<SLA> (see Table 67)


Table 52 - Service endpoint details for creating a new SLA.

URL /api/slas/

Method POST


Request body SLA (see Table 67)

Response body N/A


Table 53 showcases how one could acquire the SLA details for a particular entity. The parameters used in

the service endpoint Universal Resource Locator (URL) are provided in Table 54. Note that when the entity

type is set to “uuid”, SLARC communicates with VCG to acquire the relevant VC Tag corresponding to that

universally unique identifier (UUID). Then, the relevant SLA details are served.

Table 53 - Service endpoint details for retrieving information on an SLA of a particular entity.

URL /api/slas/by-type/{entity_type}/{entity_id}

Method GET


Request body N/A

Response body SLA (see Table 67)


Table 54 - Required parameters for properly invoking the specific-entity-SLA retrieval service.


entity_id string The entity id of interest 0xc5f496b77a1d1961d63b5bae0d81bbe809

b715f794cf2d6284906d0180e38930

entity_type string The entity type of interest vc_tag

Following the CRUD approach, the following API endpoint, specified in Table 55 and Table 56, allows the

management of specific SLAs based on their identity (ID).

Table 55 - Service endpoint details for retrieving details on or manage a SLA.

URL /api/slas/{id}





44

Request body N/A

Response body SLA (see Table 67)



Table 56 - Required parameters for properly invoking the SLA-managing service.


id string The id of the SLA 4fc0e5f9-63cc-43d1-b1ea-8c42f881d1a4

3.2.2.6 SLOs handling This set of APIs, specified in Table 57 to Table 60, allows for the proper management of CATALYST SLOs.

Table 57 - Service endpoint details for retrieving the available SLOs.

URL /api/slos/

Method GET


Request body N/A

Response body List<SLO> (see Table 66)


Table 58 - Service endpoint details for creating a new SLO.

URL /api/slos/

Method POST


Request body SLO (see Table 66)

Response body N/A


Table 59 - Service endpoint details for retrieving information on or managing a SLO.

URL /api/slos/{id}



Request body N/A

Response body SLO (see Table 66)





45

Table 60 - Required parameters for properly invoking the SLO management service.

Parameter Type Comments Example

value

id string The id of the SLO 3

3.2.3 Data Model In the following tables (Table 61 to Table 67), the data model governing the SLARC data exchanges is

documented.

Table 61 - Defaults data model.

Property Type Description Comments

slo_billing_cycle string Default SLO billing cycle

Valid input only in the range:

['daily', 'weekly', 'monthly',

'yearly']

slo_objective string The default SLO objective Valid input only in the range:

['downtime', 'brown_hosting']

slo_target_value number The default value that when

reached, the SLO is broken

slo_unit string The default units of measurement

for the value


['seconds', 'minutes', 'hours']

ctn_protocol string The default protocol of the

notification


['amqp', 'http', 'https', 'kafka']

ctn_server string The default server to send the

information to

ctn_port integer The default port to connect to

ctn_service_url string The default rest of the URL to

send the information to

ctn_percentage number

The default percentage of the

SLA's SLOs that when reached the

notification will be sent

sla_duration string The default SLA duration Valid input only in the range:

['year', 'month', 'week']

Table 62 - SLA Event data model.


id string The id of the event The format is Universally Unique

Identifier (UUID) version 4 (UUID4) [23].

type string The type of the event Valid input only in the range:


status string The status of the event Valid input only in the range: ['active',

'completed', 'inactive']

start_time string The start time of the event In ISO8601 format [24].

end_time string The end time of the event In ISO8601 format.

value number The current value of the event

as to the event type

slo string The SLO that the event refers to The format is UUID4.



46

Table 63 - The SLO levels data model.

Attribute Type Description Comments

id string The id of the level The format is UUID4.

status string The status of the level Valid input only in the range:

['active', 'inactive']

min_value number

The minimum value that

when reached, the level is

activated

N/A

max_value number

The maximum value that

when reached, the level is

deactivated

N/A

price_drop_percentage number The price drop percentage N/A

slo string The SLO that the level refers

to The format is UUID4.

Table 64 - The ClientToNotify data model.


id integer The ID of the entry The format is UUID4.

responsible string Who is the one being notified

protocol string The protocol of the notification Valid input only in the range:

['amqp', 'http', 'https', 'kafka']

server string The server to send the information to

port integer The port to connect to

service_url string The rest of the URL to send the

information to

In the case of amqp, this is the

queue to send information to. In the

case of kafka, this is the topic to

send information to.

percentage number

The percentage of the SLA's SLOs

that when reached the notification

will be sent

Multiplied by 100.

last_notified string When the last notification was sent. In ISO8601 format.

sla string The SLA to send notification for The format is UUID4.

Table 65 - The SLO billing history data model.


billing_time string The time the billing was

calculated In ISO8601 format.

start string The start time of the billing cycle In ISO8601 format.

end string The end time of the billing cycle In ISO8601 format.

value number The SLO value during this cycle

status string The SLO status during this

period

Valid input only in the range: ['ok', 'broken',

'completed']

Table 66 - The SLO data model.




47

id string The id of the SLO The format is UUID4.

objective string The SLO objective Valid input only in the range:


target_value number The value that when

reached, the SLO is broken

current_value number The current SLO value

unit string The units of measurement

for the value


['seconds', 'minutes', 'hours']

status string The SLO status Valid input only in the range: ['ok',

'broken', 'completed']

sla string The SLA to which this SLO

adheres The format is UUID4.

billing_cycle string Period for resetting the SLO

status


['daily', 'weekly', 'monthly', 'yearly']

last_billed string The last time the billing was

calculated In ISO8601 format.

next_bill string The next time the billing will

be calculated In ISO8601 format.

levels Array<Level> The List of relevant levels

events Array<Event> The list of relevant events

history Array< SLOHistory> The list of relevant billing

history objects

Table 67 - The SLA data model.


id string The id of the SLA The format is UUID4.

entity_type string The entity type to watch Normally this is

entity_id string The entity ID to watch

valid_from string The activation time of the SLA In ISO8601 format.

valid_until string The deactivation time of the SLA In ISO8601 format.

slos Array<SLO> The list of relevant SLOs



48

3.3 IT Load Marketplace Connector

3.3.1 Architecture

Figure 16 - The IT Load Marketplace Connector architecture.

The Marketplace Connector enacts the interaction of the hosting DC with the CATALYST Marketplace;

specifically, this variant, the IT Load Marketplace Connector (ITMC) with the IT Load Marketplace. This

component receives information about the candidate migrations, formulates them in a format compliant

with the CATALYST Marketplace and places them on relevant market sessions of the IT Load Marketplace.

Also, when the clearing result is available on the Marketplace side, this component will inform the affected

DCs about the result (acceptance or rejection); in case of acceptance the associated DCs will invoke the

processes for the IT Load migration, when this is planned. Finally, this component will receive information

about the migration execution result and will inform the CATALYST Marketplace accordingly.

This functionality is provided through the subcomponents, as depicted in Figure 16:

• The Northbound API exposed to the CATALYST Marketplace Information Broker (IB);

• The Southbound API exposed to the DCMC Master Client and the ITLB;

• The Market Action Manager, dealing with market actions, as understood in the CATALYST

Marketplace;

• The Migration Manager, dealing with candidate migrations, as these are received by the DCMCM.

ITMC interacts with the following components:

• The DCMCM

• The ITLB

• The IB



49

3.3.2 Interfaces Specification

3.3.2.1 Register candidate migration actions This service is meant to be used by the DCMC Master Client to register information about candidate

migrations, which will be used by the ITMC to place relevant market actions on the CATALYST IT Load

Marketplace. Specification details are provided in Table 68 and Table 69.

Table 68 - Service of ITMC for registering candidate migration actions.

URL /migration/{migration_id}/actions/{form}/

Method POST


Request body MigrationAction (see Table 79)

Response body N/A




Table 69 - Parameters required for registering candidate migration actions.


migration_id Integer The id of the migration record, as kept by the

DCMC Master Client.

1

form String The form of the marketplace on which the

market action is relevant. For the IT Load

Marketplace, this parameter should be equal

to “it_load”

it_load

3.3.2.2 Register candidate migration actions for specific marketplace timeframe This service is meant to be used by the DCMC Master Client to register information about candidate

migrations, which will be used by the ITMC to place relevant market actions on the CATALYST IT Load

Marketplace and at a market session of the timeframe provided. Specification details are provided in Table

70 and Table 71.

Table 70 - Service of ITMC for registering candidate migration actions for specific marketplace timeframe.

URL /migration/{migration_id}/actions/{form}/{timeframe}/

Method POST


Request body MigrationAction (see Table 79)

Response body N/A






50

Table 71 - Parameters required for registering candidate migration actions for specific marketplace timeframe.


migration_id Integer The id of the migration record, as kept by the

DCMC Master Client.

1


relevant market action is meant to be placed.

For the IT Load Marketplace, this parameter

should be equal to “it_load”

it_load

timeframe String The timeframe of the marketplace on which

the relevant market action is meant to be

placed (can be “intra_day”, “day_ahead”) for

the IT Load Marketplace

day_ahead

3.3.2.3 Retrieve reference prices This service is meant to be used by the ITLB in order to retrieve the current reference price of a given service

and for a given timeframe in the IT Load Marketplace. Specification details are provided in Table 72 and

Table 73.

Table 72 - Service of ITMC for retrieving reference prices.

URL /prices/{form}/{timeframe}/

Method GET


Request body N/A

Response body Price (see Table 80)


404 – No record was found matching the provided criteria


Table 73 - Parameters required for retrieving reference prices.






it_load





day_ahead

3.3.2.4 Retrieve reference prices for a given date This service is meant to be used by the ITLB in order to retrieve the reference price at a given date of a given

service and for a given timeframe in the IT Load Marketplace. Specification details are provided in Table 74

and Table 75.

Table 74 - Service of ITMC for retrieving reference prices for a given date.

URL /prices/{form}/{timeframe}/date/{date}/



51

Method GET


Request body N/A

Response body Price (see Table 80)


404 – No record was found matching the provided criteria


Table 75 - Parameters required for retrieving reference prices for a given date.






it_load





day_ahead

date Datetime The date at which the reference price is

desired in ISO8601 format

2018-02-23

T18:33:28+02

:00

3.3.2.5 Register migration status updates This service is meant to be used by the DCMC Master Client in order to inform about the migration result,

which is associated with a certain market action. Specification details are provided in Table 76 and Table

77.

Table 76 - Service of ITMC for registering migration status updates.

URL /actions/<action_id>/status/

Method POST


Request body ActionStatus (see Table 81)

Response body N/A




Table 77 - Parameters required for registering migration status updates.


action_id Integer The id of a market action in the CATALYST IT

Load Marketplace

1

3.3.3 Data Model The data model used in the services provided by ITMC is documented in Table 78 to Table 81.



52

Table 78 - MigrationAction data model.


date Datetime The time the action is placed

starttime Datetime The time on which the candidate migration is planned to start

endtime Datetime The time on which the candidate migration is planned to end

loadvalues LoadValue[] The characteristics of the load, which is planned to be migrated, or the

hardware resources offered to execute remote load

value Decimal The value of the energy equivalent of the candidate migration

uom String The unit of measurement for the (energy) value

price

Decimal The price of the market action in the CATALYST IT Load Marketplace in

Euros

action_type String The type of market action to be placed (can be either “bid” or “offer”)

Table 79 - LoadValue data model.


parameter String The name of the load characteristic (can be “cpu”, “ram” or “disk”)

value Integer The value quantifying the load characteristic

uom String The unit of measurement of the value, depending on the “parameter”.

Table 80 - Price data model.


price Decimal The reference price of given service and timeframe in Euros.

Table 81 - ActionStatus data model.


status String The status of a given market action.

3.4 Virtual Container Generator The Virtual Container Generator (VCG) is a distributed (reversed client-server model) component responsible

for persisting information related to virtual IT loads, i.e., virtual machines or containers, on distributed ledgers

and effectively transforming such loads into virtual containers (VCs). Under a federated DCs perspective, the

VCG enables IT loads traceability and allows for indisputable SLA monitoring (see section 3.2 for details).

Details about the architecture, operation and interactions of this component can be found in deliverable

D3.1 [2].

3.4.1 Interfaces On top of the interfaces implemented in the first period of the project and documented in deliverable D3.1

[2], two new interfaces have been implemented to facilitate integration with the rest of components of

CATALYST, SLARC in particular. In the following, the two new interfaces are presented.

The following endpoint, specified in Table 82, offers information on all the known VC Tags that have been

registered in the VCG blockchain infrastructure.



53

Table 82 - Service endpoint details for retrieving information on all known VC Tags.

URL /api/container/all

Method GET


Request body N/A

Response body VC Tags (see Table 85 for details)


Similarly, the following endpoint, specified in Table 83 to Table 85, allows integrated parties to query the

VCG over the VC Tag (if any) that corresponds to a VC registered in the blockchain with a UUID (e.g., UUID of

an OpenStack VM).

Table 83 - Service endpoint details for retrieving information on the VC Tag that has been registered as mapped with

a given UUID.

URL /api/container/by-id/{uuid}/

Method GET


Request body N/A

Response body VC Tags (see Table 85 for details)


404 – No VC Tag with the given UUID was found

Table 84 - Parameters required for retrieving information on the VC Tag associated with a given UUID.


uuid String The UUID of the virtual

container (e.g., OpenStack

VM)

0ee3d889-c755-440d-ad20-a7d4417cf6d6

Table 85 - VC Tags data model.


vc_tags List<string> The list of VC Tags of

interest

3.5 Energy-aware IT Load Balancing The Energy-aware IT Load Balancer (ITLB) is responsible for optimally placing IT loads over the available

hardware at federated DC level, following the energy constraints set by the IDEO in order to implement the

“follow the energy” strategy. The Energy-aware IT Load Balancing mechanism, as well as the ITLB

architecture, operation and interactions are described in D3.2 “Energy aware IT Load Balancing” [3].

The ITLB is implemented as a client-server paradigm; the client and server architectures are depicted in

Figure 17 and Figure 18, respectively, which are analyzed in D3.2. The ITLB client is in charge of identifying



54

and selecting available DC resources able to receive ITL from other DCs participating to the federation.

Basically, the ITLB client identifies the candidate ITLs for being migrated and the available hardware in DCs

for hosting remote loads, feeding the process of formulating and posting ITL migration bids and offers to the

IT Load Marketplace. ITLB operation, at client and federated level, is described in section 2.3 of D3.2.

Figure 17 - Energy aware IT Load Balancer - Client architecture.

Figure 18 - Energy aware IT Load Balancer - Server architecture.

At the federation level, the ITLB (server) is acting as the clearing mechanism for the IT Load Marketplace.

This component is responsible for the application of placement algorithm based on vector bin packing

strategy [25]. This component is deciding which ITL goes in which DC. The results are then transferred to the

DCMC client which is responsible for the actual migration management. The details of the placement

algorithm are available in section 5 of D3.2. The complexity of the algorithm is described in section 5.5. It is



55

demonstrated that beyond the heuristics applied to lower effective complexity, the real-world application will

consider a relatively reasonable number of DCs, and thus placement possibilities, from IT perspective.

3.5.1 New Interfaces The interfaces of ITLB have been introduced in D3.1. Table 86 includes the new interfaces defined in order

to support the federated DC migration concept.

Table 86 – New interfaces of ITLB Client

Provided Interfaces

Migration

Description Update migration, mainly status

Provided to DCMC Master Client

End-point URL /migration/

Request body {

"vc_tag":

"vdfkbjafhvlafjvendflvkdsfnvjndsflvkshdl",

"status": "rejected"

}

Allowed Methods POST

Required Interfaces

End-point URL

MigrationRequest

Provided by DCMC Master Client

3.6 Federated DC Migration Process View In this section, the migration process is further elaborated, identifying interactions among the CATALYST

components at a finer level of technical detail. The sequence diagram in Figure 19 depicts interactions

among the ITLB (Client), DCMC (Master Client), IT Load Marketplace Connector (ITMC), and Information

Broker (IB), in line with the Federated DC Migration Process Design.

Moreover, sequence diagram in Figure 20 depicts the process view of the Federated DC Migration Process.



56

Figure 19 - IT load migration sequence diagram.



57

Figure 20 - The process view of the Federated DC Migration Process.



58

4 IT Load Balancing and Migration manual In this section the installation and user’s guidelines are presented.

4.1.1 Virtual Container Generator The source code of the component is available at the ‘Releases’ public subgroup of the “H2020 CATALYST”

group of GitLab [26]. The installation guidelines as well as the software and hardware resources required to

operate the VCG are detailed in deliverable D3.1 [2].

4.1.2 Energy-aware IT Load Balancer The source code of the component is available at the ‘Releases’ public subgroup of the “H2020 CATALYST”

group of GitLab [27]. The installation guidelines as well as the software and hardware resources required to

operate the ITLB are detailed in deliverable D3.2 [3].

4.1.3 SLA (Re-)negotiation The code of SLARC is available at the ‘Releases’ public subgroup of the “H2020 CATALYST” group of GitLab

[28]. The software requirements of SLARC are minimal, only requiring an operational Docker engine and

docker-compose installation, its version being at least 17.09.0. As such, no specific requirements related to

operating systems or other software are in place. For details as to how docker and docker-compose are

meant to be installed, the interested user is requested to refer to [29].

4.1.3.1 Configuration and installation In order to install the component, one has to download the code from the aforementioned link and then

perform the commands listed in Figure 21. The configuration options useful for the third step of the

installation instructions are provided in Table 87.

# Go to the SLARC source code directory

$ cd <download_directory>/slarc

# Go to the docker directory of the SLARC source

$ cd docker

# Edit the configuration file (see Table 87 for details)

$ vim .env

# Build and start SLARC

# In some systems, the command might need to be granted with superuser privileges

# Depending on the host operating system, the modification of the following command

# might change.

$ docker-compose up --build -d

Figure 21 - Installation instructions for SLARC.



59

Table 87 - SLARC configuration options.

Variable Default Description

API_PORT 80

The port the service should be listening to. For

production purposes, this should change to 443 and

the code should be appropriately managed to use

HTTPS with valid SSL certificates. This should be

performed at load balance level (see Table 88 and

Table 89 for discussion)

CELERY_POSTGRES_HOST celery-

database

The tasks State DB service name. It is advisable not to

change this setting.

CELERY_POSTGRES_DB celery The State DB database name.

POSTGRES_HOST database The State DB service name. It is advisable not to

change this setting.

POSTGRES_DB slarc The State DB database name.

POSTGRES_USER slarc The State DB username. It is advisable to change this

setting.

POSTGRES_PASSWORD slarc The State DB password. It is advisable to change this

setting.

REDIS_HOST redis The Cache service name. It is advisable not to change

this setting.

RABBITMQ_HOST rabbitmq The Tasks Queue Manager service name. It is

advisable not to change this setting.

RABBITMQ_USER user The Tasks Queue Manager service name. It is

advisable to change this setting.

RABBITMQ_PASSWORD password The Tasks Queue Manager service name. It is

advisable to change this setting.

RABBITMQ_VHOST / The Tasks Queue Manager service vhost name.

SLA_CHECK_PERIOD */2

The frequency (in minutes) of checking for SLA

breakages, in crontab format. Default is every two

minutes. See [30] for details and discussion. Changing

the default “minutes” behaviour might imply changes

in the base SLARC API settings file, available in

slarc/slarc/settings/base.py.

VCG_API_SERVICE N/A The VCG API service base URL.

After successful configuration and building of the SLARC software stack, eleven (11) containers should have

been spawned, all being named after the convention catalyst_slarc_<component_name>_1, as

depicted in Figure 22.

Figure 22 – Components of a typical SLARC installation.

In Table 88, the mapping of the SLARC microservices containers to the SLARC architecture is presented.



60

Table 88 - Mapping of the SLARC instance containers against the component architecture.

Container name Component Comments

catalyst_slarc_tasks-dashboard_1 Tasks Manager A simple dashboard to overview ongoing SLA-

measuring tasks

catalyst_slarc_api_1 Northbound/Southbound API The core API-implementing component

catalyst_slarc_scheduled-tasks-

manager_1 Tasks Manager Manager for handling scheduled tasks

catalyst_slarc_task-manager-1_1 Tasks Manager Manager for real-time tasks (1/3)



catalyst_slarc_redis_1 Cache Used for tasks coordination and DB

management speedup

catalyst_slarc_rabbitmq_1 Task Queue Manager

Used as a message exchange mechanism

between the API and the Task Managers. Can

also be used as a notification mechanism.

catalyst_slarc_database_1 State DB Used by the API components

catalyst_slarc_celery-database_1 State DB Used by the Task Managing components

catalyst_slarc_nginx_1 Northbound/Southbound API Part of the API, effectively implementing load

balancing and static files serving

In Table 89 the ports needed for proper SLARC operation are presented. Note that only port 80 is normally

needed, unless if configured in a different way through the SLARC environments configuration file.

Table 89 - Ports exposed by SLARC.

Port Optional Component Description of use

80 No Load Balancer

Exposes the basic API functionality of SLARC, load

balances requests among the running instances of the

core API containers

5555 Yes Tasks Manager Dashboard depicting the operation of the Task Manager,

showcasing active tasks, their status etc.

15672 Yes Task Queue Manager Dashboard overviewing the operation of the Task Queue

Manager, useful to understand the load of SLARC.

5672 Yes Task Queue Manager

Used for exposing the Tasks Queue Manager

functionality to other components. Useful in the cases

where the SLA breakage notification should be

implemented using the AMQP protocol and exposed

through the internal Tasks Queue Manager instance.

Note that by default, SLARC is listening to a plain HTTP port, no support for SSL encryption being provided

by this prototype. It should be noted, however, that HTTPS connections should be employed in a production

environment, normally exposing port 443.

4.1.3.2 SLARC synchronization with existing VCG instances The SLARC operator is able to initialize itself by synchronizing with an existing VCG instance by issuing the

command of Figure 23.



61

# In some systems, the command might need to be granted with superuser privileges

# Depending on the host operating system, the modification of the following command

# might change.

$ docker exec -it catalyst_slarc_api_1 pipenv run python manage.py sync_entities

Figure 23 - Synchronizing existing VC Tags from a running VCG instance.

4.1.4 IT Load Marketplace Connector The following instructions for ITMC deployment assume that the following requirements are met:

• OS: Ubuntu Server 16.04;

• SW: Docker and Docker-compose support.

No special hardware requirements have been identified for ITMC deployment and operation.

The first release of the ITMC source code can be downloaded from the public Gitlab project “IT Load

Marketplace Connector” [31].

Before building the container, the ITMC must be configured to point to the current IB and DCMC deployment.

To do this, the .env file must be edited, by updating the values of the MARKETPLACE_IB_BASE_URL and

DCMCM_URL variables with the IP address where the IB and DCMC listen to. Also, the OAUTH_URL must be

set to the IP address of the authentication server of DCMC server. Moreover, the marketplace credentials of

the host DC must be configured, in order to allow participation in the IT Load Marketplace, by editing the

MARKETPLACE_USERNAME and MARKETPLACE_PASSWORD parameters.

After the code has been downloaded, the following commands must be run:

$ cd itl-marketplace-connector/config/docker

vim .env ## To set the Marketplace-related settings, DCMCM_URL, OAUTH_URL

$ bash launch.sh

Upon successful deployment, ITMC can be reached at:

http://{itl_marketplace_connector_ip}/

4.1.5 Federated DC Migration Controller The following instructions for DCMC deployment assume that the following requirements are met:

• OS: Ubuntu Server 16.04;

• SW: Docker and Docker-compose support.

The first release of the DCMC source code can be downloaded from the public Gitlab project “Federated DC

Migration Controller” [32].

4.1.5.1 DCMC Server Installation guidance on the DCMC Server can be found on the public GitLab project Federated DC Migration

Controller Server [33], which is always updated as per current developments. In the following subsections,

information about the prerequisites for the DCMC Server and installation instructions are provided.

4.1.5.1.1 Prerequisites

For the deployment of the DCMC Server component the Docker engine as well as docker-compose should

be installed. These actions can be performed following the instructions provided below. Firstly, an update

should be performed, and essential packages should be installed:



62

$ sudo apt-get update

$ sudo apt-get install -y \

apt-transport-https \

ca-certificates \

curl \

software-properties-common

Secondly the key and Docker repository should be added:

$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

$ sudo add-apt-repository \

"deb [arch=amd64] https://download.docker.com/linux/ubuntu \

$(lsb_release -cs) \

stable"

Then another update is performed, Docker is installed, and the user is added to docker group.

$ sudo apt-get update

$ sudo apt-get install -y docker-ce

$ sudo groupadd docker

$ sudo usermod -aG docker $USER

Finally, docker-compose should be installed:

$ sudo curl -L

"https://github.com/docker/compose/releases/download/1.24.0/docker-compose-

$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

$ sudo chmod +x /usr/local/bin/docker-compose

4.1.5.1.2 Deployment

The DCMC Server is deployed as collection of Docker containers, utilizing docker-compose. Having cloned

the code of this repository, and, having created the .env file, the following commands should be executed:

$ cp .env /dcmc-server

$ cp .env /dcmc-server/dcmc_server

$ cp .env /dcmc-server/dcmc_server/settings

$ cd dcmc-server

$ docker-compose -p 'catalyst' up --build -d

4.1.5.2 DCMC Master Client In the following, the essentials for the installation of a DCMC Master Client instance are described, including

prerequisites for the deployment and the actual deployment. Detailed instructions can be also found online,

on the public GitLab project “Federated DC Migration Controller Master Client” [34].



63


For the deployment of the DCMC Master Client the Docker engine as well as docker-compose should be

installed. These actions can be performed following the instructions provided in section 4.1.5.1.1.


The DCMC Master is deployed as collection of Docker containers, utilizing docker-compose. Having cloned

the code of this repository, and, having created the .env file, the following commands should be executed:

$ cp .env /dcmc-master-client

$ cp .env /dcmc-master-client/dcmc_master_client

$ cp .env /dcmc-master-client/dcmc_master_client/settings

$ cd dcmc-master-client


4.1.5.3 DCMC Lite Client In the following, the essentials for the installation of a DCMC Lite Client instance are described, including

prerequisites for the deployment and the actual deployment. Detailed instructions can be also found online,

on the public GitLab project “Federated DC Migration Controller Lite Client” [35].


For the deployment of the DCMC Lite Client, the Docker engine as well as docker-compose should be

installed. These actions can be performed following the instructions provided in section 4.1.5.1.1.


The DCMC Lite is deployed as a single Docker container, utilizing docker-compose. Having cloned the code

of this repository, and, having created the .env file, the following commands should be executed:

$ cp .env /dcmc-lite-client

$ cd dcmc-lite-client


It should be noted that the DCMC Lite Client is normally launched automatically during the preparation of

the migration when the VContainer which will host the OpenStack Virtual Compute Node is created. Hence,

these installation instructions are given mostly for reference and/or testing.



64

5 Conclusions In this document, the CATALYST approach to live migration of virtual loads has been presented. The

presented approach is innovative, as live migration is enabled among DCs, which belong to different

administrative domains, without affecting the end users’ accounting. Thus, it is achieved with minimum

downtime and it is transparent to the end user (DC customer). On the other hand, live migration is not solely

provided as a technical solution in the CATALYST project, but it is part of a well-defined procedure, framed

by concrete B2B relations among DCs, with terms and conditions agreed/defined by IT Load Marketplace

interactions. Thus, CATALYST provides an integrated solution to DC Managers, enabling the consideration of

IT load migration as a source of flexibility within the smart grid, in which they participate, and, eventually, as

an additional source of revenue. In other words, the CATALYST Federated DC Migration effectively integrates

modern IT industry with smart energy grids, leading to sustainable DC ecosystems in economic,

environmental and operational terms.

The document provides a general overview allowing the interested reader to comprehend the flow of

operations leading to live migration, within the CATALYST ecosystem. Moreover, technical description of

involved components accompanies the software prototype, which is provided as open source in the “H2020

CATALYST” Gitlab group [4]. As a next step, the presented prototypes will be integrated in the CATALYST

framework, providing the means of executing flexibility actions related to IT load migration, as suggested by

the CATALYST optimization module.



65

References

[1] CATALYST, "D2.3 - Final CATALYST Framework Architecture," H2020 -768739 CATALYST Deliverable,

2019.

[2] CATALYST, "D3.1 - VC Generation and Migration," H2020 -768739 CATALYST Deliverable, 2019.

[3] CATALYST, "D3.2 - Energy aware IT Load Balancing," H2020-768739 CATALYST Deliverable Report,

2019.

[4] Gitlab, "H2020 CATALYST Gitlab Group," 2019. [Online]. Available: https://gitlab.com/project-

catalyst. [Accessed 2019].

[5] CATALYST, "D2.1: CATALYST Specification & Design," H2020-768739 CATALYST Deliverable Report,

2018.

[6] OpenStack, "OpenStack," 2019. [Online]. Available: https://www.openstack.org/.

[7] CATALYST, "D5.1 - CATALYST market infrastructure implementation," H2020-768739 CATALYST

Deliverable Report , 2019.

[8] CATALYST, "D5.2 - Optimal orchestration of CATALYST market mechanisms," H2020-768739

CATALYST Deliverable Report, 2019.

[9] OpenStack, "Keystone," 2019. [Online]. Available:

https://www.openstack.org/software/releases/stein/components/keystone.

[10] OpenStack, "OpenStack Compute (nova)," 2019. [Online]. Available:

https://docs.openstack.org/nova/latest/.

[11] R. Sturm and W. Morris, Foundations of Service Level Management, Pearson, 2000.

[12] "Django REST framework," [Online]. Available: https://www.django-rest-framework.org/. [Accessed

Aug. 2019].

[13] "Celery: Distributed Task Queue," [Online]. Available: http://www.celeryproject.org/. [Accessed Aug.

2019].

[14] "PostgreSQL Home page," [Online]. Available: https://www.postgresql.org/. [Accessed Aug. 2019].

[15] "Redis," [Online]. Available: https://redis.io/. [Accessed Aug. 2019].

[16] "RabbitMQ," [Online]. Available: https://www.rabbitmq.com/. [Accessed Aug. 2019].

[17] "AMQP - Advanced Message Queueing Protocol," [Online]. Available: https://www.amqp.org/.

[Accessed Aug. 2019].

[18] "NGINX | High Performance Load Balancer, Web Server, & Reverse Proxy," [Online]. Available:

https://www.nginx.com/. [Accessed Aug. 2019].

[19] "Docker: Enterprise Container Platform for High-Velocity Innovation," [Online]. Available:

https://www.docker.com/. [Accessed Aug. 2019].

[20] Apache Kafka, "Apache Kafka: A distributed streaming platform," Sep. 2019. [Online]. Available:

https://kafka.apache.org.

[21] "OpenAPI Specification (version 3.0.2)," [Online]. Available: https://swagger.io/specification/.


[22] "Swagger," [Online]. Available: https://swagger.io/. [Accessed Aug. 2019].

[23] P. Leach, M. Mealling and R. Salz, "RFC 4122 - A Universally Unique IDentifier (UUID) URN

Namespace," 07 2005. [Online]. Available: https://tools.ietf.org/html/rfc4122.html. [Accessed 08

2019].

[24] ISO, "ISO 8601 DATE AND TIME FORMAT," 2019. [Online]. Available: https://www.iso.org/iso-8601-

date-and-time-format.html. [Accessed 08 2019].

[25] "Vector Bin Packing, David S. Johnson, 2014, Department of Computer Science, Columbia University,

New York, NY, USA, Encyclopedia of Algorithms, DOI 10.1007/978-3-642-27848-8_495-1".



66

[26] CATALYST, "Virtual Container Generator," GitLab, 2019. [Online]. Available:

https://gitlab.com/project-catalyst/releases/virtual-container-generator.

[27] CATALYST, "IT Load Balancer," GitLab, 2019. [Online]. Available: https://gitlab.com/project-

catalyst/releases/it-load-balancer.

[28] CATALYST, "SLA Renegotiation Controller," GitLab, 2019. [Online]. Available:

https://gitlab.com/project-catalyst/releases/slarc.

[29] Docker Inc., "Install Docker Compose," . [Online]. Available:

https://docs.docker.com/compose/install/. [Accessed Aug. 2019].

[30] Celery Project, "Periodic Tasks," [Online]. Available:

https://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html#crontab-schedules.


[31] CATALYST, "IT Load Marketplace Connector," GitLab, 2019. [Online]. Available:

https://gitlab.com/project-catalyst/releases/itl-marketplace-connector.

[32] CATALYST, "Federated DC MIgration Controller," GitLab, 2019. [Online]. Available:

https://gitlab.com/project-catalyst/releases/federated-dc-migration-controller.

[33] CATALYST, "Federated DC Migration Controller Server," GitLab, 2019. [Online]. Available:

https://gitlab.com/project-catalyst/releases/federated-dc-migration-controller/dcmc-server.

[34] CATALYST, "Federated DC Migration Controller Master Client," GitLab, 2019. [Online]. Available:

https://gitlab.com/project-catalyst/releases/federated-dc-migration-controller/dcmc-master-client.

[35] CATALYST, "Federated DC Migration Controller Lite Client," GitLab, 2019. [Online]. Available:

https://gitlab.com/project-catalyst/releases/federated-dc-migration-controller/dcmc-lite-client.

Federated DCs Migration Controller - Project Catalyst · 2020. 4. 15. · d3.3 federated dcs...

Documents

Transcript of Federated DCs Migration Controller - Project Catalyst · 2020. 4. 15. · d3.3 federated dcs...