20-May-2003HEPiX Amsterdam EDG Fabric Management on Solaris G. Cancio Melia, L. Cons, Ph. Defert, I....

20-May-2003 HEPiX Amsterdam

EDG Fabric Management on Solaris

G. Cancio Melia, L. Cons, Ph. Defert, I. Reguero,J. Pelegrin, P. Poznanski, C. Ungil

Presented by M. Guijarro.

CERN/IT


Fabric management on Solaris.• EDG: the European DataGRID project

• Fabric Management (WP4)

• Global functioning of installation and configuration

• Configuration

• Installation

• Node Configuration Management

• Solaris port

• Status and plans


EDG

• http://cern.ch/eu-datagrid

• DataGrid is a project funded by European Union. The objective is to build the next generation computing infrastructure providing intensive computation and analysis of shared large-scale databases, from hundreds of TeraBytes to PetaBytes, across widely distributed scientific communities.

http://cern.ch/eu-datagrid


EDG• Divided in Work Packages:

– Middleware: • WP1: Work Scheduling• WP2: Data Management• WP3: Monitoring services• WP4: Fabric Management• WP5: Storage Management• WP6: Integration Testbed & Support• WP7: Network

– Applications:• WP8: Particle Physics• WP9: Earth Observation• WP10: Biology


Fabric Management (WP4)

• Divided in Tasks:– Installation– Configuration– Monitoring– Fault Tolerance– Resource Management– Gridification– Integration

}

Today's subject


WP4: Global idea

CCM

SPMASPMANCMComponents

Cdispd

NCM

RegistrationNotification

SPMA

SPMA.cfg

CDB

nfshttp

ftp

Mgmt APIACL’s

Client Nodes

SWRep Servers

cache Packages(rpm

, pkg)

packages

(RPM, PKG)

PXEDHCP

Mgmt APIACL’s

Installation server

DHCPhandling

KS/JS

PXEhandling

KS/JSgenerator

NodeInstall

CCM

Node (re)install?

EDG group slide


WP4: Configuration and Installation

• Objective:To develop system management tools for enabling the deployment of very large computing fabrics […] with reduced sysadmin and operation costs.

• Installation task: solutions for – automated from scratch node installation– node configuration/reconfiguration– software storage, distribution and installation

• Configuration task: solutions for – storing, maintaining and retrieving configuration

information.


WP4: ConfigurationCentral Configuration Database (CDB):

• Common store for configuration information– …including what software packages to deploy from which repository on which

nodes

• Configuration information can be arranged in templates:

• Possible to create template combinations/hierarchies to match service structures

• Each template can be maintained (using a GUI) by a different person

• Configuration information is validated and kept under version control using transactions

LXBATCHlxbatch444

LinuxBase

packagesCC

packagesEDG/LCGm/ware

lxbatch445lxbatch446


WP4: Installation

• The Software Package Management and Distribution subsystem is responsible for managing and storing software packages, and the distribution and installation of these packages on client nodes.

• SWrep (Software Repository):– Software modules are bundled into packages using a given packaging

format, like RPM for most Linux distributions, or PKG for Sun/Solaris.– The packages themselves are stored on a managed software

repository. – This repository is accessible via protocols like HTTP, FTP, or a shared

file system.

• The Node Configuration Management provides a framework for adapting the actual configuration of a node to its desired configuration, as it is described in the node’s profile inside the CDB.– This target information is made available to the node via a configuration

component running on each node.– Node components are notified by a daemon which is polling the CDB.


WP4: NCM

● NCM – Node Configuration Management

● Client software running on the node which takes care of “implementing” what is in the configuration

● Configurations are centrally stored, managed and accessed (CDB), using XML profiles (per node)

● “Components” (like SUE features) are responsible for updating local config files, and notifying services if needed


WP4: Global idea

CCM

SPMASPMANCMComponents

Cdispd

NCM

RegistrationNotification

SPMA

SPMA.cfg

CDB

nfshttp

ftp

Mgmt APIACL’s

Client Nodes

SWRep Servers

cache Packages(rpm

, pkg)

packages

(RPM, PKG)

PXEDHCP

Mgmt APIACL’s

Installation server

DHCPhandling

KS/JS

PXEhandling

KS/JSgenerator

NodeInstall

CCM

Node (re)install?

EDG group slide


Solaris: CDB

• Configuration Database– CDB stores the hardware and software configuration in

a configuration server – PAN is used to compile HLD to LLD– Clients (NCM,AII) access the CDB using the Node

View Access API (Configuration Cache Manager)

• Solaris– Global Schema has to be adapted to Solaris– PAN, CDB and CCM already ported


Solaris – AII

• Automated Installation Infrastructure– Installs machines acording to the configuration in the

CDB– 3 modules:

• DHCP• NBP (Network Bootstrap Program)• OS Installer

• Solaris– Loader is PXElinux in Linux -> OpenBoot in Solaris– Anaconda/Kickstart in Linux -> Jumpstart in Solaris


Solaris: SPMA

• SPMA– Reads the list of installed packages (OS)– Get the list of packages to be installed (CDB)– Computes the differences– Determines the list of operations to do– Calls the package installer/de-installer

• Solaris port– Reads pkg data base (-> rpm)– Pkgt installs/de-installs (-> rpmt)– ASIS apps packages with pkg (-> rpm)


Solaris: Implementing SPMA

spma-target.cf

<PATH> <PRODUCT> <VERSION> <RELEASE> <ARCHITECTURE>Example:/afs/.cern.ch/asis/PKGS/sun4x_58 ASIS-ASIS-applog 1.4 0 sun4x_58

...

spma-managed-packages

<PRODUCT> <VERSION> <RELEASE> <ARCHITECTURE>

Example:- ASIS-ASIS-applog 1.4 0 sun4x_58

...

differences List of operations

check_conflicts

store_actions

arrange_actions

execute_actions

pkgt

SPMA is an application relying on a set of libraries, all system independent, except the Packager class (virtual) which is inherited by the platform dependent class (SysVPkgr for Solaris)

PKG Transactions is a tool to install, upgrade and remove Solaris packages in one transaction.


Solaris: NCM

• Node Configuration Manager– Updates the configuration of the machine when the

configuration in CDB changes– Provides the framework, components are needed for

the different local services

• Solaris– Many linux components reusable (if designed with

portability in mind)– Some specific components needed (30% of the current

SUE features are Solaris-specific)


Solaris: WP4 Status

• Installation– ASIS applications have been packaged with pkg– SPMA and pkgt work but are still in test– NCM in detailed design phase (collaboration with

EDG, more focused on Linux)– AII in design phase

• Configuration– Pan, database and cache manager ported and

included in CVS– Global schema worked on (design validated)


Solaris: WP4 Future

• 2003/end Q2: LCG-1 with WP4/Linux installation and configuration (already ~ 100 nodes)

• 2003/end Q3: SPMA and pkgt will be used for Solaris 9 certification at CERN

• 2003/end Q3: SPMA and pkgt deployed in the Computer Centre

• 2003/end Q3: CDB on Solaris.

• 2003/end Q4: GUI for a CDB editor


Conclusion

• Specific Resource:– J. Pelegrin (CERN - SPMA, pkgt, ASISdist)– C. Ungil (SUN fellow – CDB, AII, NCM)– S. Lopienski (CERN – CDB editor)

• Proof of concept of WP4

• Proof of portability (RedHat 386, Solaris but also could be ia64)

20-May-2003HEPiX Amsterdam EDG Fabric Management on Solaris G. Cancio Melia, L. Cons, Ph. Defert, I....

Documents

Transcript of 20-May-2003HEPiX Amsterdam EDG Fabric Management on Solaris G. Cancio Melia, L. Cons, Ph. Defert, I....