The Path to Java: Writing, Implementing and Using APIs 5.18.2011

Robert Hyerleresource

Hewlett-Packard Company

Writing, Implementing and Using the Java API's

The Path to SAF Java:

Friday, April 29, 2011

History-- how did we get here?

Understanding-- why this way?Using

-- how does all this play out?

Implementing-- OpenSAF, and other approaches

Going Forward-- what’s needed next?


History-- how did we get here?

Hewlett-Packard—circa 2005—We needed to deliver events (locally) from a Java application to a C++ WEBM CIMOM. I didn’t want to invent an API. I had some acquaintance with SAF. The Event Service API fit the bill, except that it didn’t have a Java version. I read through the spec. Then I got out my IDE (vi) and wrote the Java mapping based on 3 basic interfaces/object types and simple data types/structures. Everything else followed, more or less.


The Event Service mapping in SAF remains pretty much the same today:

Service Handle Channel Events

is a factory for .. is a factory for, producer of ..

.. and so does the mapping style that has been used for the other mappings. If you understand the mapping style, you’ll understand the mappings.

We’ll get to that shortly. But, first, a little more history ..


The HP Event Service API was built and deployed. Note that we never built or used an Event Service: the API provided a single, local channel (via named pipes) from the Java Application to the C++ CIMOM.

We settled on:• object/interface factory style• use of interfaces vs. use of classes• exceptions instead of return values• naming for packages, types, and parameters• leverage of Java types and memory management

We did not address:• callbacks and the “selection object”• bootstrapping: associating implementations with service interfaces

History


HP later abandoned WBEM for telecom software management. We adopted SAF IMM and NTF. The first deployments of this new management framework were largely Java based. So, we defined IMM and NTF Java mappings.At this point, we had to address callbacks, selection, and bootstrapping.These mappings were contributed to SAF for standardization. The approach—style and tight correspondence with the C specification—were adopted by SAF and used when mapping the Cluster Service and Availability Management Framework.

History

Event API

HP NTF & IMM API’sand

Implementations

SAF Java Standard with Cluster and

AMF as well


It wasn’t only HP!Ericsson had been working on Java mappings and on Java standardization as well. In particular, Ericsson was addressing:-SAF services/interfaces used in containerized applications,

esp. J2EE-Providing higher level interfaces. These can built on top of

the lower level mappings, hiding them from the application developer.-Bridges to other Java management standards, esp. JMX

History

HP ran into difficulties with containerized applications and the collaboration on the various aspects of Java mapping proved beneficial to everyone.


Understanding-- why this way?

The Java mappings use JavaDoc as the “specification language”. Let’s take a look!

send

void send() throws AisLibraryException, AisTimeoutException, AisTryAgainException, AisBadHandleException, AisInvalidParamException, AisNoMemoryException, AisNoResourcesException, AisVersionException, AisTooBigException, AisUnavailableExceptionThe notification is sent. Objects referenced by this Notification should not be altered between the point the send() is invoked and it returns. If a notification ID has been allocated for this notification, it is used. Otherwise, a new notification ID is provided. Many of the operations that are performed during notification "allocate" in the C API are performed during the send and this may give rise to exceptions not found in the C API version of "send."SAF Reference: saNtfNotificationSend, saNtfNotificationSendWithId

Throws:AisLibraryExceptionAisTimeoutExceptionAisTryAgainExceptionAisBadHandleExceptionAisInvalidParamExceptionAisNoMemoryExceptionAisNoResourcesExceptionAisVersionExceptionAisTooBigExceptionAisUnavailableExceptionSee Also:Notification.getNotificationID()

Kind of sucks.


3.16.10 saNtfNotificationSend() and saNtfNotificationSendWithId() PrototypeSaAisErrorT saNtfNotificationSend( SaNtfNotificationHandleT notificationHandle);SaAisErrorT saNtfNotificationSendWithId( SaNtfNotificationHandleT notificationHandle, SaNtfIdentifierT notificationIdentifier);ParametersnotificationHandle — [in] The handle which was obtained by a previous call to one of the saNtf<notification type>NotificationAllocate*() functions and which identifies this particular notification instance. The SaNtfNotificationHandleT type is defined in Section 3.14.1.2 on page 48.notificationIdentifier — [in] The notification identifier to be assigned to the notification to be sent. This notification identifier must have been previously allocated by invoking the saNtfIdentifierAllocate() function, and it must not already have been used previously to send a notification successfully. The SaNtfIdentifierT type is defined in Section 3.14.11 on page 54.DescriptionThese functions are used to send a notification. The notification is identified by the notification handle that is returned in the notification structure created with a preceding call to one of the saNtf<notification type>NotificationAllocate*() functions.

The C specification has five pages of explanation for send (only the first is shown here).

You can’t simply use the JavaDoc to understand the API:-You have to understand the C specification (and have a

copy next to you)-You need to put on your Java “spec. writer glasses”


UnderstandingThe Spec Writer Glasses

-There is only one API specification: the one based on C.-The Java mapping is at the same semantic level and with the

same semantics.-It is not required, but it must be a straightforward exercise

to implement the Java API by wrapping an existing C implementation. This wrapper must be thin (minimum state, no threads).-“C errors” present in the specification may pop up in Java.


UnderstandingStart with the C spec, then ...- ditch all the memory allocation and reuse mechanisms- forget all those C arrays with auxiliary variables to

describe the valid portions, allocated size, etc.- all the status return values go away: they become

exceptions. Either the function becomes void, or, the now available return value slot is used for something more interesting.- the names become shorter (take advantage of package

scope)- primitive Java types are often used for parameters- we have half the number of type definitions and half the

number of parameters


The anchor points

There is a shared package: org.saforum.ais and it is actually quite a bit different from the ais.h file. There are very few shared type definitions. The contents center on service factories, the dispatch mechanism, and exceptions. Unlike the C spec’s, there is some implementation code in this package!

There is one (root) package per service. In each, find the handle type (an interface). Its name is derived from the name of the service: e.g. for the Event Service (EVT), the name is EvtHandle. Now find the handle’s factory: in this example: EvtHandleFactory (it’s a class extending FactoryImpl). Everything else follows!


From the handles, find the associated objects defined using interfaces.

Unless there is sufficient complexity/functionality to warrant it, parameters to methods are usually Java primitive types/classes, simple structures, or arrays. No “getter/setter” or other encapsulation style is imposed.

Function pointers are mapped as interfaces with a single method. So the callback structures have fields of the interface type (objects). The single method of that object can then be invoked.

Understanding


In depth:finding the implementation(of the client library)

Understanding

- An application should not need to know (or even be able to know?) about the specific SAF implementation; should not be able to control which implementation is used.- The surrounding environment needs to be able to deploy/launch an

application with the desired SAF implementation.- For certain situations—notably testing—being able to select an

implementation is desirable.

Version version = new Version('A', (short) 4, (short) 1); NtfHandle.Callbacks callbacks = new NtfHandle.Callbacks(); callbacks.notificationCallback = new OnNotificationCallback(); NtfHandleFactory factory = new NtfHandleFactory(); try { handle = factory.initializeHandle(callbacks, version); Consumer consumer = handle.getConsumer();

} catch (AisException e) { e.printStackTrace(); // acceptable for an example, but not for production }

for example:


- Use the Java system properties and security manager.- The implementation class—

expressed as properties defining the class name and URL—is loaded via the URLClassLoader.- The security manager controls

which properties an application can read and write.- There is a rather complicated

relation between the shared factory implementation and those of each service.

/* * set up the URL classloader. Note that we tolerate a null URL--converting it to a * bogus URL--since the URL may never be used. */ URL[] path = new URL[1]; path[0] = new URL((implClassURL == null) ? "http://bogus-impl-class.url.com/" : implClassURL); ClassLoader loader = new URLClassLoader(path,Thread.currentThread().getContextClassLoader());

/* * load the implementation class using the URL classloader to find it only if it is not found * by the current classload chain. */ Class<? extends Object> implClass = loader.loadClass(implClassName);

public FactoryImpl() { /* * obtain the keys from the extending class */ implClassNamePropertyKey = getImplClassPropertyKey(); implClassURLPropertyKey = getImplURLClassLoaderPropertyKey(); /* * using the keys, obtain the implementation class name * and location */ implClassName = System.getProperty(implClassNamePropertyKey); implClassURL = System.getProperty(implClassURLPropertyKey); }

finding the implementation


In depth:dispatch();

Understanding

- There are two methods of determining if a callback is pending:- via the “legacy” Java mapping hasPendingCallback() method- via the use of java.nio channels which closely follows the C standard- HP used the hasPendingCallback() method before “nio” existed. The

implementation was an RMI call to the area server which returned when a callback was pending.

public void doOne() { try { handle.hasPendingCallback(10000); // 10 usec's handle.dispatch(DispatchFlags.DISPATCH_ONE); } catch (AisException e) { e.printStackTrace(); // acceptable for an example, but not for production } return; }

old way:


In depth:dispatch(); public void listenNIO() throws IOException, AisException {

SelectableChannel channel; // obtained from our service handle Selector selector; // only selecting on one channel here SelectionKey key; Set<SelectionKey> keys;

channel = handle.getSelectableChannel(); channel.configureBlocking(false); selector = Selector.open(); key = channel.register(selector, SelectionKey.OP_READ);

while (true) { if (selector.select() == 0) continue; // false alarm? keys = selector.selectedKeys(); keys.remove(key); // need to clear this to avoid loop try { handle.dispatch(DispatchFlags.DISPATCH_ALL); } catch (AisTryAgainException e) { continue; } catch (AisTimeoutException e) { continue; } catch (AisBadHandleException e) { break; // must have shutdown } }

new way:

easy way: public void doOne() { try { handle.dispatchBlocking(10000); } catch (AisException e) { e.printStackTrace(); // acceptable for an example, but not for production } return; }

unfortunate choice of name

how can this fail? let us count the

ways!


Using-- how does all this play out?

An example using NTF:- marshaling data- callbacks- exceptions

Producer- initialize service and

pre-allocate notifications- send notifications in

response to internal events- shutdown

Consumer- initialize service and set

up callbacks- receive notifications- shutdown


Using

private Producer producer; private ClassId classId; private ServiceUser serviceProvider; private SecurityAlarmDetector securityAlarmDetector;

/** * Set up the Notification service for the GateKeeper. All notifications will have the * same ClassId, ServiceUser, and SecurityAlarmDetector; defined here. Other fields * in the notification will depend on the specific security violation. * @see com.example.notifications.NotificationWrapper#initialize() */ public void initialize() { Version version = new Version('A', (short) 2, (short) 1); NtfHandle.Callbacks callbacks = new NtfHandle.Callbacks(); NtfHandleFactory factory = new NtfHandleFactory(); try { handle = factory.initializeHandle(callbacks, version); producer = handle.getProducer(); } catch (AisException e) { e.printStackTrace(); // acceptable for an example } classId = new ClassId(); classId.vendorId = 33333; classId.majorId = 995; classId.minorId = 1;

serviceProvider = new ServiceUser();serviceProvider.value = new SafClientUtils.ValueStringImpl("switch configurator");securityAlarmDetector = new SecurityAlarmDetector();

securityAlarmDetector.value = new SafClientUtils.ValueStringImpl( "com.example.notifications.GateKeeper.checkAccess");

}

Initialize & pre-allocate

producer


Using

public void alertAccess(String user, String role, String object, String operation) {ServiceUser serviceUser = new ServiceUser();

serviceUser.value = new SafClientUtils.ValueStringImpl(user); Date now = new Date(); try { SecurityAlarmNotification alarm = producer.createSecurityAlarmNotification(

EventType.OPERATION_VIOLATION, object,

classId, now.getTime() * 1000 * 1000, // SAF uses nanoseconds, not milliseconds ProbableCause.AUTHENTICATION_FAILURE, Severity.SEVERITY_MAJOR, securityAlarmDetector, serviceUser, serviceProvider);

alarm.setAdditionalText("Access Denied!"); alarm.send(); } catch (AisException e) { e.printStackTrace(); // acceptable for an example, but not for production } }

Send notifications

producer


Using

public void shutdown() { if (handle == null) return; // already shutdown, or tried to long timeout = 500; // sleep timeout milli's if library is not responding int attempts = 3; // max tries to shutdown, double timeout each attempt do { try { handle.finalizeHandle(); } catch (AisLibraryException e) { break; // library dead, can we report this?, give up here } catch (AisTimeoutException e) { try { Thread.sleep(timeout); } catch (InterruptedException e1) { // ignore this, just try again } timeout <<= 1; // wait longer next time if we try again continue; } catch (AisTryAgainException e) { continue; // might work next time? } catch (AisBadHandleException e) { // don't bury this, it's a bug! re-throw! throw new IllegalStateException("Expected the notification handle to always be valid",e); } } while (--attempts > 0); handle = null; // indicates we're done, and let's the garbage collector go to work }

Shutdown

producer


public void initialize() { Version version = new Version('A', (short) 4, (short) 1); NtfHandle.Callbacks callbacks = new NtfHandle.Callbacks(); callbacks.notificationCallback = new OnNotificationCallback(); ClassId classIds[] = new ClassId[] { new ClassId() }; classIds[0].vendorId = 33333; classIds[0].majorId = 995; classIds[0].minorId = 1; Severity severities[] = new Severity[] { Severity.SEVERITY_MAJOR, Severity.SEVERITY_CRITICAL }; NtfHandleFactory factory = new NtfHandleFactory(); try { handle = factory.initializeHandle(callbacks, version); Consumer consumer = handle.getConsumer(); NotificationFilters filters = new NotificationFilters(); filters.securityAlarmFilter = consumer.createSecurityAlarmNotificationFilter( null, (FilterName []) null, null, classIds, null, severities, null, null, null); consumer.createSubscription(filters); // we drop the return value } catch (AisException e) { e.printStackTrace(); // acceptable for an example, but not for production } }

Initialize

consumer


/** * A SecurityAlarmNotification callback that just prints some basic information. */ class OnNotificationCallback implements NotificationCallback {

@Override public void notificationCallback( Subscription subscription, ConsumedNotification notification) { try { SecurityAlarmNotification alarm = (SecurityAlarmNotification) notification;

// and if the cast fails? Date eventTime = new Date(alarm.getEventTime() / (1000*1000)); // convert to milliseconds System.out.println( "Security Alarm at " + eventTime.toString() + ", with severity " + alarm.getSeverity() + ", type: " + alarm.getEventType() + ", probable cause: " + alarm.getProbableCause() + " (" + alarm.getAdditionalText() + " " + alarm.getLocalizedMessage() + ")"); } catch (AisException e) { e.printStackTrace(); // acceptable for an example, but not for production } } }

callbacks

consumer

Using


/** * Listen for security alarms until we shutdown this GateWatcher. * This method blocks and will only return after {@link #shutdown()} is called. * Hence, the caller should provide a separate thread to make this call. */ public void listen() { try { handle.dispatch(DispatchFlags.DISPATCH_BLOCKING); } catch (AisException e) { e.printStackTrace(); // acceptable for an example, but not for production } return; }

Receiving Notifications

consumer

Using

Shutdown:no difference between consumer and producer!


Implementing-- OpenSAF, and other approaches

JNI wrappersBecause of the close semantic levels of the C standard and the Java mappings (plus other considerations when the mappings were designed), using the Java Native Interface as the glue between the Java and C API’s is feasible. This is the OpenSAF approach.


Implementing public interface ClusterMembershipManager {

ClusterNode getClusterNode( int nodeId, long timeout ) throws AisLibraryException, AisTimeoutException, AisTryAgainException, AisBadHandleException, AisInvalidParamException, AisNoMemoryException, AisNoResourcesException;

public class ClusterMembershipManagerImpl implements ClusterMembershipManager {

public native ClusterNode getClusterNode(int nodeId, long timeout) throws AisLibraryException, AisTimeoutException, AisTryAgainException, AisBadHandleException, AisInvalidParamException, AisNoMemoryException, AisNoResourcesException;

/************************************************************************** * FUNCTION: Java_org_opensaf_ais_clm_ClusterMembershipManagerImpl_getClusterNode * TYPE: native method * Class: ais_clm_ClusterMembershipManager * Method: getClusterNode * Signature: (IJ)Lorg/saforum/ais/clm/ClusterNode; *************************************************************************/JNIEXPORT jobject JNICALL Java_org_opensaf_ais_clm_ClusterMembershipManagerImpl_getClusterNode( JNIEnv* jniEnv, jobject thisClusterMembershipManager, jint nodeId, jlong timeout ){ // call saClmClusterNodeGet _saStatus = saClmClusterNodeGet( _saClmHandle, (SaClmNodeIdT) nodeId, (SaTimeT) timeout, &_saClusterNode );

org.saforum.ais.clm.ClusterMembershipManager.java:

org.opensaf.ais.clm.ClusterMembershipManagerImpl.java:

${INSTALL}/java/ais_api_impl_native.clm.j_ais_clm_manager.c:

no code!


ImplementingNative JavaWhen HP built an IMM implementation, there were no C implementations. As well, we had a Java application framework (SmartFrog) that lent itself to IMM. So, we built the area server, the client libraries, and the transport with standard Java plus a replicated database.

Mixed ImplementationsHP has also prototyped C client libraries communicating with Java area servers using a neutral protocol: JSON-RPC

database

Java IMM

Java Client Lib

database

Java IMM

rmi

active/standby

application


Going Forward-- what’s needed next?

Apply the recipe to the Log ServiceThis is an important service for a wide range of applications

Keep the Java mappings up to date with changes in the underlying specifications

Complete the Java implementations in OpenSAF!


The Path to Java: Writing, Implementing and Using APIs 5.18.2011

Technology

Transcript of The Path to Java: Writing, Implementing and Using APIs 5.18.2011