Post on 31-Mar-2015
An Evaluation of Contemporary Commercial
SOAP Implementations
Presented by: Alex Ng
Alex NgDepartment of Computing,
Macquarie University
Shiping Chen CSIRO ICT Centre
Paul Greenfield CSIRO ICT Centre
AWSA 2004, Melbourne, 13-14 April 2004
Outline Background Experimental design Result discussion Related work Future work
Web Services Definition (W3C)A Web service is a software system
designed to support interoperable machine-to-machine interaction over a network. It has an interface described in a machine-processable format (specifically WSDL). Other systems interact with the Web service in a manner prescribed by its description using SOAP-messages, typically conveyed using HTTP with an XML serialisation in conjunction with other Web-related standards. W3C Web Services Architecture Working Draft (8 August
2003)
W3C Web Services Conceptual Stack
HTTP,HTTPS,SMTP
XML,SOAP,WS-Addressing
XSD,WSDL,UDDI,Policy
Security Reliable Messaging
Transaction
Composition Services Assurances
Description
Messaging
Transport
BPEL4WS
Service Composition
Source: Ferguson D. Secure, Reliable, Transacted Web Services: Architecture and Composition. MSDN technical article, Sept. 2003.
SOAP (Simple Object Access Protocol)
Simple Extensible Based on XML De facto
standard for XML Messaging in supporting Web Services
V1.1(May/00) V1.2 (June/03)
Request POST /StockQuote HTTP/1.1Host: www.stockquoteserver.comContent-Type: text/xml Content-Length: nnnnSOAPAction: “Some-URI”<SOAP:Envelopexmlns:SOAP="urn:schemas.xmlsoap.org:soap.v1"><SOAP:Header>
<t:Transaction xmlns:t=”URI”mustUnderstand=”1”>5</
t:Transaction></SOAP:Header><SOAP:Body>
<m:GetLastTradePrice xmlns:m="URI">
<symbol>DIS</symbol></m:GetLastTradePrice>
</SOAP:Body></SOAP:Envelope>
Other Competitors of SOAP Web Services
Use different transport mechanism Microsoft DIME (Direct Internet Message
Encapsulation) Work with or without SOAP & HTTP send attachments of various types
BEEP (Blocks Extensible Exchange Protocol) Connection oriented, min 257 channels,
asynchronous message exchange Work with or without HTTP
Use different Web Architecture REST (Representational State Transfer)
An architectural style (Roy Fielding) Use XML/HTML, no need for SOAP
Performance Issues of SOAP Web Services
Serialisation and de-serialisation Message size ( 6-8 times larger than
binary) Introduce latency in transmission, storing or
retrieving XML data
XML Processing Parsing (encode/decode XML elements) Validation (conformation to a pre-defined schema) Transformation (XML <-> other format)
Network settings Issues HTTP 1.1 chunking (avoids content-
length calculation) HTTP 1.1 Keep-Alive (reduces connection
set up cost) HTTP 1.1 Pipelining (handles multiple
requests/connection) TCP-Delayed-ACK (defers receiver acknowledgement)
Nagle Algorithm (defers the sending of small packets)
Evaluation of contemporary commercial SOAP implementations
(1) What level of performance is shown by major commercial SOAP Web Services implementations?
(2) How does the performance of SOAP compare to that offered by conventional non-SOAP technologies?
Document/Literal RPC/Literal
Document/Encoded
RPC/Encoded
SOAP Body
Document RPC
SOAP Parameter
Literal
Encoded
Experimental Design
Test Driver
Platform Specific Runner
Property.xml
Test Document
Result Log
Target Application
Server
Target Web Service
100Mbps Ethernet
4x1.6Ghz, 3.6Gb
4x1.6Ghz, 3.6Gb
Invoice: One Customer, 50 product lines
Complex Message
20 customers InquiryMedium Message
Single customer inquiry/update
Simple Message
Implementation Doc/ Lit
Doc/ Enc
RPC/ Lit
RPC/ Enc
Product A
Product B
Product C
Encoding
StyleSimple Messag
e
Medium Messag
e
Complex
Message
Product A
Doc/Lit 873 7581 21392
Doc/Enc 1593 13973 38842
RPC/Enc 1542 13969 38838
Product BDoc/ Lit 870 7513 21566
RPC/Enc 1597 14035 38999
Product C RPC/Enc 1664 14566 40273
No.of Char.
Result: Message Size
Result: Latency
Result: Simple Message Throughput
0
200
400
600
800
1000
1200
1400
4 16 32 64 128
Concurrent Threads
MP
S
Prd-A Doc/ Enc Prd-A Doc/ Lit Prd-A RPC/ Enc Prd-C RPC/ Enc
Prd-B Doc/ Lit Prd-B RPC/ Enc Prd-R
Result: Medium Message Throughput
0
100
200
300
400
500
600
4 16 32 64 128
Concurrent Threads
MP
S
Result: Complex Message Throughput
0
50
100
150
200
250
300
350
4 16 32 64 128
Concurrent Threads
MP
S
0
500
1000
1500
2000
2500
3000
3500
4000
4500
5000
Document/Literal RPC/Encoded
mS
Client Deserialisation
Network receive
Server Serialisation
Server Application
Server Deserialisation
Network send
Client Serialisation
Result: Serialisation/De-serialisation
23%
13%
30%10%
Evaluation Summary
SOAP Performance is affected by: Encoding style (Document/Literal encoding is the
choice) Product of implementation (a sizable gap between two
products using the same document/literal encoding) Performance delivered by the majority SOAP
implementations: Simple Message – able to deliver reasonably good performance (4
to 6ms vs 1ms non-SOAP) Medium Message – (10 to 22ms vs 7ms non-SOAP) Complex Message – (20 to 63ms vs 15ms non-SOAP)
De-serialisation overhead is higher than that of serialisation.
Related Work (1) The Extreme Lab, Indiana University
Govindaraju, et. al. sending different size of linked list of doubles using
different implementations: SOAPRMI, SUNRMI, and NEXUSRMI, and TCP.
TCP is two orders of magnitude better than SOAPRMI Serialisation and de-serialisation are the bottlenecks.
Chiu, et.al. sending different arrays of mathematical doubles. Suggested Optimization: Pull Parser, schema-specific
XML parser, trie and HTTP 1.1. Kohlhoff and Steele: Compares SOAP against FIX
and CDR SOAP latency is 2 to 3 times worse than CDR FIX is a text based encoding and outperforms CDR and
CDR
Related Work (2) Dhawan: Compares the performance of .NET
Remoting against ASP.NET ASP.NET XML serialisation is more efficient than .NET
Remoting SOAP serialisation
Davis and Parashar Different SOAP Implementations: SOAPRMI/Java1.1,
Microsoft SOAP Toolkit SP2 in VB, Perl SOAP::Lite, Apache SOAP 2.2, and Apache Axis Alpha 3.
inappropriate TCP options had caused 200ms delays in some implementations (MS SOAP, Apache SOAP and SOAP:Lite )
Conclusion: SOAP is in the orders of magnitude slower than JavaRMI and CORBA
Related Work (3) Benkner, et. al.: Compares the performance
of JAXM implementations against JAXRPC in different package sizes (1x1024 -> 8x128) Different Implementations: Apache Axis 1.1
RC2, Sun JWSDP 1.1, Mind Electric GLUE4.0, Systinet WASP 4.5, Novell jBroker Web 2.0, and X-Treme XSOAP 1.2.20.
Conclusion: JAXM & JAXRPC produce similar results Best performance result is achieved when using
large packages of data, i.e. splitting 1MB of source data into four packages of 256KB each for JAXM in Systinet WASP implementation and 2x 512 for JAXRPC in Apache implementation.
Future Work Extend Evaluation to other SOAP
Implementations, such as Apache Axis
Identify other factors when SOAP over HTTP is used across wide-area networks
Your Feedback
Reference Chiu K., et. al. (2002) Investigating the Limits of SOAP Performance for
Scientific Computing In Proceedings of 11th. IEEE International Symposium on High Performance Distributed Computing HPDC-11 2002 (HPDC'02)
Benkner, S., Brandic, I., Dimitrov, A., et al. Performance of Java Web Services Implementations. In Proceedings of ICWS'03. Las Vegas, 2003
Davis, D. and Parashar, M. Latency Performance of SOAP Implementations. In Proceedings of IEEE Cluster Computing and the GRID 2002 (CCGRID'02)
Dhawan, P. Performance Comparison: Exposing Existing Code as a Web Service, October 2001 (on-line) Accessed Date: 10 October 2002http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnbda/html/bdadotnetarch11.asp
Kohlhoff, C. and Steele, R. (2003) Evaluating SOAP for High Performance Business Applications: Real–Time Trading Systems. In Proceedings of WWW2003. Budapest, Hungary.
Govindaraju, M., A. Slominski, et al. Requirements for and Evaluation of RMI Protocols for Scientific Computing. Supercomputing. IEEE, 2000