SOA Quality Assurance

Distributed Environment Testing Strategies, Issues and Best Practices

SOA – Services Oriented Architecture

SOA is the journey to an elusive destination.Service orientation is all about agility, reuse and business focus The meaning of the term is dualistic: It is both abstract business and technical at the same time. A continuum between philosophy and code.It is not a specific technology or product, but a collection of enabling technologies, standards and products put together as a SOA

Semantic layer provides business representation of data

User Lifecycle Administration

User Administration

Resource Access Control

Event Management

A Solution – Service Oriented Architectures

SOA is an approach to organizing and using

IT to match and combine needs with

capabilities in support of the overall mission

of an enterprise

Capabilities performed by one for another to achieve a desired outcome

Functionally aligning architecture to enable a collection of independent services to be linked together to

solve a business problem

The fundamental organization of a system embodied in its capabilities, their interactions, and the environment

Architecture

Oriented

Service

SOA - A paradigm that encourages organizations to re-think how their IT capabilities are organized

Difficult decisions

Model-Based Concurrent Engineering Processes

Vision

it’s about

Of course strategy is hard

making tough choices.

SOA Quality Assurance Testing Problem

Distributed Environment Redefines QADistributed = Evolving Distributed Computing SystemsNew software relies on old running systemsTesting environment cannot be controlled

Scale testing a special problemRequires new relationships and responsibilities for Service Producers and Service ConsumersRequires a SOA Core Service Broker

Service Level AgreementsCore Enterprise Services, Registry and Metadata RepositoryAccreditation and Certification Services

What is SOA?

The nexus between IT and Business – allow for a common dialog

An new approach to both IT and Business gets done

ReuseAgilityEfficiencyLoosely-CoupledSharing

SOA is NOT ...WebServices, Enterprise Service Bus, etc.

SOA tools and software are NOT a panaceaSOA Infrastructure software is NOT a replacement for sound distributed systems engineeringSOA tools will NOT address required Social EngineeringSOA tools and Infrastructure software will NOT be universally and consistently understoodGet help from your vendors and understand that most SOA tools can be used cooperatively

Message-Oriented Middlewareasynchronous != loosely coupledUnderstand the differences between Broker-based ESB and MOM-based ESB

NewA Methodology

What is SOA not?

SOA is not a Thing

It’s not even an Architecture … it’s just an approach toward an architecture (of which distributed systems architectures are the most applicable).

It’s a way of thinking and working such that the product of the work results in a systems that is:

Agile and AdaptableEngenders reuseEncourages and Enables Sharing of Existing Systems and Data

The SOA Products offerings only provide a toolbox of technical solutions, not a “silver bullet.”

SOA is not a Methodology

SOA nor SOA tools DO NOT do Distributed Engineering for you

ESB does not solve ALL design problemsMOM or EAI solutions are engineering solutions for specific Distributed Engineering problems

The only thing harder than designing and building a complex distributed system is TESTING and DEBUGGING them

SOA GovernanceI’m not sure what this is really it. Nor do I know anyone else who has a really good answer.

“The Nexus between politics and operations”

Comprised of Policies, Procedures and MetricsService Lifecycle

Governance Tools:UDDI RegistryMetadata RepositoryWebServices Management (SOA Software [BlueTitan], AmberPoint, etc.)Traditional Enterprise Management Systems (Tivoli, HP OpenView, BPM Patrol, etc.)

Governance usually optimized for one of several outcomes:Reuse – The Cornerstone of Reuse is CommunicationAgilityPositive Financial OutcomeSharing

SOA Governance is just beginning to mature“Draconian”, “Autocratic”, “Oligarchy” or “Facist” Governance does not work well in large organizationsUnderstanding how to govern large

Usual System Testing

Development EnvironmentContinuous IntegrationUnit and Integration Testing

QA EnvironmentProtected environmentScale TestingHardware identical to Production EnvironmentSeparate network

Production EnvironmentSame as QA EnvironmentHandles multiple Security Enclaves (NIPR, SIPR, JWICS)

Issues with Distributed SOA TestingIt impossible to perform traditional QA testing a Distributed Environment

Multiple Producer supported environments DevelopmentUnit-TestScale-TestingProduction

Secure enclave testing required standalone serviceProducers must provide packaged service for SCIF based developmentLike to use VMWare or other Hypervisor technology to reduce technology burden on consumer.What happens if the service is a composite service?Even with SCIF based development, final scale and functional testing wants to use the provider-hosted service

Issues with Distributed SOA Testing

There is no “Global” synchronized clockAll event correlation required use of cause-event based clocks – otherwise known as Lamport Clocks.Unclear how to do this is a large-scale coordinated way

Auditing and LoggingConsistently available common logging and auditing services are required.Should be provided by a shared service infrastructure (NCES?)

NCES does not list this as a common service nor a segment of Enterprise Service Management

Adding Friction

Producer FrictionProducer Accreditation and CertificationProducing “Composite” Services

Consumer FrictionUsage friction – adding hurdles to utilizing shared data and servicesVetting consumers in various degreesRoot-case analysis problems with provided composite services

Responsibility of Service Providers

Provide the service using a “Community Standard” interfaceWebServices: XML, XSD, HTTP, SOAP, etc.JMS, FTP, SMTPNOT: MQ Series, .NET, Proprietary “extensions” to Standards

Provide an Accredited ServiceAssert that the services lives up to “Total Assurance or Service Safety” policies and standards

Register the Service with the Shared Service Infrastructure providerHandle the Unintended User?

Responsibility of Service Consumers

Use the service the way the producers intended it to be usedAccess service via brokered channel

Access via Shared Service Infrastructure as opposed to direct accessNo “unintended” Denial-of-Service attacksNo oversized payloadsNo unreasonable SLA expectations

Open QA resultsReport errors promptlyUtilize auditing, logging, security, etc. where possible

Role of the Broker-based InfrastructureProvide the Authoritative Registry and Metadata Repository

Registry for Run-timeRepository for Design-time

Provide the Accreditation and Certification ServiceFunctionalSecurity / IA (Information Architecture)Service Level Agreement Ranges

Provide SLA Adjudication ServicesConcept: SLA in the “Eye of the Consumer”

Provide Metrics publishing for GovernanceProvide Scalable Service NetworkProvide automatic “Testing Mode” switching

Conclusion

SOA is a dangerously overused acronym that required specific definition in each organization Your Organization with SOA done correctly will display the emergent properties of Agility, Reuse, Efficiency, Loosely-Coupling, etc.Functional Testing (QA) is brutally difficult to do in an open Distributed Computing EnvironmentGovernance from Broker, Provider and Consumer is not a solved problem

Most challenges in Your Organization service creations and usage will depend greatly on how Social Engineering and Communication is accomplished