Distributed Handler Architecture

Beytullah Yildizbyildiz@indiana.edu

Web Service Handler

• Additive functionality to Web Services.• Incrementally adds new capability to Web Service

endpoint.• Supports more modular architecture; separation

of tasks.• Processes SOAP header and body.• Called as either handler or filter.• Many handlers can get together to build a chain.

2

Motivations I

• Web Services become fat because of utilizing many handlers.

• A handler may cause a convoy effect.• Requirements for distributing handlers.• Requirements for efficient handler

orchestration.

3

Motivations II

• Reusability– A handler may be utilized by many Web Services.

• Modularity– Clean separation of the tasks improves

modularity.

• Loosely coupling – Messaging decouples the computing nodes ,

handlers and endpoints.

4

Research Issues I

• Scalability.– Adding new handlers should not degrade the system

performance.

• Performance.– What are the benefits and costs of the distributed handler

structure?

• Flexibility– A new handler should be easily deployed and removed.

• Orchestration of the Distributed Handlers.– Efficient and effective handler workflow structure

5

Research Issues II

• Messaging for the distributed handlers. –What are advantages and disadvantages ?

• Parallel execution of the handlers.–What are advantages and disadvantages ?

• Architectural principles for the distributed handlers.–Conditions for the distribution of a handler.

6

Motivating Scenario I-A

• A typical handler execution scenario is sequential execution.

• The cost of the sequential handler execution : Tlogger+ Tmonitor + Tconverter millisecond

7

Motivating Scenario I-B

8

• The cost of the concurrent handler execution : MAX(Tlogger, Tmonitor, Tconverter)+Toverhead millisecond

Motivating Scenario II

• A CPU bound handler, having convoy effect on the execution, is deployed to a faster machine.

• Handler A9

gf10 3638 millisecond

everest 696 millisecond

Difference 2942 millisecond

Distributed Handler Architecture (DHArch)

10

Gateway• An Interface between DHArch and A Web Service Container.• Provides flexibility to deploy DHArch for different SOAP processing engines.•Necessary to be written for each native system.

11

The advantages of using NB in DHArch

• Queuing regulates message flow.• Provides guaranteed message delivery.• Efficient.• Scale vey well; tree structure. – Many handlers can be addressable.

• Asynchronous messaging.

12

Messaging Format

• XML document.• Consists of three main

parts:– ID

• UUID generated key.

– Properties• Carries the necessary

properties to the handler.

– Payload• Carries the message.• SOAP message.

13

<context><id>123456789</id><properties>

<oneway>true</oneway></properties><payload>

The message</payload>

</context>

Communication Manager

14

Highlights of the Execution• DHArch utilizes two context objects:– Native container context – Distributed Handler Message Context

• Two level orchestration prevents the orchestration engine to become too complex.

• Queues are leveraged to regulate the message flow.

• Caching idea is utilized to expedite the message processing by decreasing the access time.

• The execution queue can be prioritized.

15

Distributed Handler Message Context

• Keeps necessary information about a message to carry out the execution.• Every message has its own context.• Flow structure is kept inside of the context.• Basically, stores

• The message• Orchestration structures• Handler related parameters• Stage related parameters.

16

Two level Orchestration

17

• Separation of the flow description and the execution

• Four constructs in description level:• Sequential• Parallel• Looping• Conditional

• Engine utilizes only two execution types:– Sequential– Parallel

18

Orchestration Schema

19

Execution

Message Journey between two stages• A message travels from stage to stage• Stages has the ability to execute handlers in parallel manner.• A message can not exit from a stage without completing the execution of its handlers•Every stage contains at least one handler.•Every flow execution contains at least one stage.

20

Test I- Performance I

4.

5.

6.

21

Handler A CPU Bound

Handler B CPU Bound

Handler C IO Bound

Handler D CPU/IO

Handler E IO Bound

1. Axis in-memory2. Handlers are sequential in DHArch 3. Handler are parallel in DHArch

Stage 1 A, C

Stage 2 E

Stage 3 B,D

Stage 1 A, B

Stage 2 E

Stage 3 C,D

Stage 1 A,B,C,D

Stage 2 E

• Results are gathered in four environments:• Multicore• Multiprocessor• Local Area Network• Single processor

• Five handlers are utilized.• Six configurations are created.

Test I- Performance II

22

Test I- Performance III

23

Test II- Scalability I

• Results are collected in four environments:• Multicore• Multiprocessor• Local Area Network• Single processor

• The elapsed time of the service is measured while the number of the handlers is increasing.

• The same handler is utilized.• The handlers are running sequentially.• Single broker is leveraged.

24

Test II- Scalability II

25

Test II- Scalability III

26

Test III-Cost I• Results are collected in four environments:

• Multicore• Multiprocessor• Local Area Network• Single processor

• We measured the overhead of adding a single handler.• We executed the same number of handlers both in

typical Axis handler mechanism and in DHArch . • The formula is :

– Difference = elapsed time in DHArch – elapsed time in regular Axis deployment

– Overhead = Difference/ the number of handlers

27

Test III-Cost II

28

Test III-Cost III

29

Principles for distributing a handler

• Performance wise, a handler should provide enough contribution to be distributed.– Tgain ≥ Toverhead

• Some handlers cannot be separated from the Service Endpoint because of their nature. – i.e. Reliability

• Some handlers may not be efficiently distributed for every environment.– i.e. Security in WAN requires additional security.

30

Contributions• A generic architecture for the efficient distributed

handler execution.• Leveraging queuing system for the handler

distribution.• Ability to utilize distributed resources for handler

execution.• Ability to update the handler chain during

execution.• Two level handler orchestration .• Parallel handler execution.• Detailed performance evaluation of Distributed

Handler Architecture.31

Questions and Comments ?

32