IRMIS Components Overview Gabriele Carcassi 18 April 2011.

IRMIS Components Overview

Gabriele Carcassi

18 April 2011

IRMIS3 Modules

IRMIS has been conceptually divided into independent modules• Components (installation/cabling/lightweight inventory),

Lattice, eLog, PV, …

Motivation:• Easier to have independent development• User can pick and choose

This talk is about the Components module

IRMIS3 DATA MODEL

IRMIS3 Data Model - Components

Where is it?

How is it powered?

How is it controlled?

How is it connected?


Distinguishes between• Component instance - physical component with inventory

information associated with it• Component installation - set of instructions of how a

particular component is installed (housed, powered, controlled, cabled, …)

Independent concepts (can track an instance in the inventory without having it installed; can track an installation without caring which instance)


Each component is of a particular type• The type defines attributes: manufacturer, ports, which

parents/children can have, generic key/value pairs (e.g. pointers to documentation, maximum drawn current, local contact, …)

Each component installation• Lives in multiple hierarchies: control, housing and power• Can be cabled• Can be associated with a Field Name (i.e. ioc name, site

naming convention) and a set key/value pairs (i.e. MAC address, geolocation)


Instances of each type can be tracked in 3 ways:• NONE: components of that type are not tracked• COUNT: only the number of components is tracked• INSTANCES: each instance is tracked individually

- is identified by a serial number which has to be unique within the same type (“N/A” is only non-unique allowed): use barcode if available

- can have multiple key/value pairs (i.e. last inspection date, purchase order number, …)


Cable type defines what type of cable (copper, fiber, diameter, …) and how is it used (high power, BPM, …)

Each cable connects two ports of two component instances

Support for cable bundle/pull• Each bundle/pull includes a set of cables• Each bundle/pull can be associated with a path of different

components (i.e. this bundle goes from this horizontal tray, to this vertical tray, …)

IRMIS3 SERVICE ARCHITECTURE

Project infrastructure

Production server since January 2009

“alpha” - AuthZ and AuthN only designed

Build and test serverAutomated tests

IE or firefox on Windows/Linux/Mac supported

SourceForge project (mailing list, bug tracking, wiki).Monthly release cycle.

IRMIS3 Architecture

IRMIS DB(MySQL)

Data service

Javaapplets

AJAXcomponents

Scripts and CLI(perl, Python, …)

Browser(Firefox, IE, …)Web server

(Glassfish)

Data service enforces business rules(clients can be dumb)

Data service allows db schema orbusiness rule changes

Data service provides a good abstractionlayer on which to integrate

Software Stack

XML protocol (REST style WS)

Few

dat

abas

e ut

ilitie

s: b

acku

p,

cons

iste

ncy

chec

k, e

tc…

Clie

ntS

erve

r

Inte

grat

ion

with

ext

erna

l to

ols

(i.e.

phy

scs)

3rd p

arty

Per

l/Pyt

on

scrip

ts

3rd p

arty

Jav

a ap

ps J

ytho

n sc

ripts

Web applications

JavaScript bridgeApplets and

Widgets

Java Client API

Data Service layer

Database layer

Getting data

<componentType id="90" description="Oscilloscope: 4 Channel @ 500MS/s; 500MHz BW" name="HP54540A"> <manufacturer id="5" name="Agilent (HP)"/> <formFactor id="5" description="Freestanding"/> <functions> <function id="46" description="CCMS"/> <function id="8" description="Instrument"/> </functions> <requires> <interface id="143" description="GPIB_Slave" relType="control"/> <interface id="5" description="Freestanding" relType="housing"/> <interface id="78" description="120VAC" relType="power"/> </requires> <presents> <interface id="234" description="Port" relType="control"/> </presents> <ports> <port name="Chnl 1"> <portType id="13" name="BNC-F" group="RF Connectors"> <pinDesignator id="303" name="1"/> </portType> <pin usage="Chnl 1" pinDesignatorId="303"> <signalType id="1" direction="IN"/> </pin> </port> <port name="Chnl 2">...</port> <port name="Chnl 3">...</port> <port name="Chnl 4">...</port> <port name="RS232">...</port> ... </ports></componentType>

This is an example of what the data service might deliver

XML returnedREST style (data service)

Id references

De-normalized data

Writing data<transaction xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xmlns='http://xml.bnl.gov/schema/irmis' xsi:schemaLocation='http://xml.bnl.gov/schema/irmis irmis.xsd'> <create> <manufacturer name="Acopian"/> </create> <update> <componentType id="2" name="Site" description="Overall facility. Root of housing hierarchy."> <manufacturer id="1" name="None"/> <formFactor id="1" description="Virtual"/> <properties/> <functions/> <requires/> <presents/> <ports/> </componentType> </update> <delete> <formFactor id="4" description="IndustryPack"/> </delete> <connect> <cable color="blue" label="AA" portAId="123" portBId="321"/> <conductor pinAId="123" pinBId="321"/> <conductor pinAId="124" pinBId="322"/> </connect> </transaction>

XML describing a transaction sent through a POST

allows remote/disconnected work

no lock “FOR UPDATE”

idempotent actions (error or same result)

different independent “commands”

Implementation

MySQLJPA

JAXB

SAXBusiness

logic

Servlet

Read

Write

JAVA SE 6JAVA EE 5GLASSFISH 3.X

Stream out XML as soon as possibleto reduce delay

JAXB to convert transaction requestto Java objects

JPA vs JDBC: simpler queries to maintain6% performance loss

Performance

Dell Precision M4300 laptop, Intel Core Duo T7500, 2.20 GHz - APS dataset• Entire component data set streamed in about 5 seconds (31338

components, 63911 relationships)• Most other queries under 100 ms (manufacturers 88ms, interfaces

95ms)

Roughly half of the time is spent in DB access and half in XML generation• Streaming starts right after DB access

Only tuning done is on the database query (was “good enough”)• Other areas: hardware/os tuning, xml generation improvements,

profiling, caching, …

IRMIS3 CLIENT API

Java API: design principles

The Java API implement the XML protocol and provides objects that represent the data

All reads are cached: no need to keep caches in your code.

All reads are synchronized• you can have reads on multiple threads on the same data

and will not trigger multiple reads on the server (extremely useful for GUIs)

Java API design principles

The read access is object oriented• Data is exposed through interfaces

- Component, ComponentType, Manufacturer

• Some classes function as “finders” for the classes containing the data- Components.getById(123), FormFactors.getByName(“Rack”)

• You can follow references from one object to the other- component.getComponentType().getManufacturer()

Java API design principles

The write access is protocol oriented• You create a transaction• You call methods to add elements in the transaction

- Manufacturers.createManufacturer(transaction, “ACME”)

• You save the transaction, which may fail• The read access is refreshed only after the data is saved

Importing components from Excel (using Jython)

from gov.bnl.irmis.api import *from java.net import URIfrom java.io import Filefrom jxl import *

workbook = Workbook.getWorkbook(File(“components.xls"))sheet = workbook.getSheet(0)

nRows = sheet.getRows()nColumns = sheet.getColumns()

irmisDB = IrmisDB.getInstance();irmisDB.setURI(URI("http://cs-build.nsls2.bnl.gov:8080/IRMISDataService/data"))

transaction = irmisDB.createTransaction();for row in range(1, nRows): componentTypeName = sheet.getCell(0, row).getContents(); componentType = ComponentTypes.getByName(componentTypeName); serialNumber = sheet.getCell(1, row).getContents(); Components.createComponent(transaction, componentType, serialNumber)transaction.save();print "Operation successful"

Importing from Excel (Jython)






Load excel file







Connect to the service







Loop over the rows







Get data outCreate components

Fluent API

Navigating through hierarchies can be tedious On top of the “beans” API there is a fluent API

(internal domain specific language)

Search API

Builders

Change Set API

IRMIS3 USER INTERFACE

UI design

Web interface: uses applets and Web Start (both greatly improved with Java 6u10)

Uses Swing standard components plus open source (SwingX, Flamingo, JIDE, …)

UI elements can optionally be used in custom pages and integrated with javascript

UI elements

Component hierarchy tree widget (filtering, sorting, search, …)

Component pathwidget

Component applications

Component inventory

Cabling applications

Status

Component and Component types applications is considered production level

Cabling UI is considered beta while the API and service support is considered production

IRMIS Components Overview Gabriele Carcassi 18 April 2011.

Documents

Transcript of IRMIS Components Overview Gabriele Carcassi 18 April 2011.