04 October 2007Kaiser: COMS W4156 Fall 20071 COMS W4156: Advanced Software Engineering Prof. Gail...
-
Upload
merry-houston -
Category
Documents
-
view
218 -
download
0
Transcript of 04 October 2007Kaiser: COMS W4156 Fall 20071 COMS W4156: Advanced Software Engineering Prof. Gail...
04 October 2007 Kaiser: COMS W4156 Fall 2007 1
COMS W4156: Advanced Software Engineering
Prof. Gail Kaiser
http://york.cs.columbia.edu/classes/cs4156/
04 October 2007 Kaiser: COMS W4156 Fall 2007 2
Reprise: Web Services
• Web Services = distributed applications, services and components, described using XML-encoded WSDL interfaces, that process XML-encoded SOAP messages
• XML, SOAP and WSDL constitute baseline specifications that provide a foundation for application integration
04 October 2007 Kaiser: COMS W4156 Fall 2007 3
But…
• Additional standards beyond this baseline become necessary as WS applications become more complex, integrating multiple components across multiple organizations
• Otherwise, WS developers are compelled to implement higher-level functionality in proprietary and non-interoperable ways
• Solution?: WS-* Set of Specifications
04 October 2007 Kaiser: COMS W4156 Fall 2007 4
Example: Department Store Chain Enterprise
Application Integration• Background: The chain discovered that different
credit approval applications had been developed in various parts of the company.
• Solution: The chain exposed one credit approval application as a Web Service. They linked this to their point-of-sale, warehouse and financial applications.
04 October 2007 Kaiser: COMS W4156 Fall 2007 5
Example: Department Store Chain Enterprise
Application Integration• Business Benefits: The chain was able to use
the same credit approval application with the three distinct applications. As a result:
• Credit approvals became more consistent • Maintenance costs decreased
04 October 2007 Kaiser: COMS W4156 Fall 2007 6
Tier 1 -- Enterprise Application Integration
• Companies initially use Web Services to integrate internal applications
• Web Services allow them to expose legacy applications in heterogeneous environments without having to rewrite significant amounts of code
04 October 2007 Kaiser: COMS W4156 Fall 2007 7
Example: Car Rental Company Interoperability
with Key Partner• Background: A major airline approached the
car rental company about putting a link to the car reservation system on the airline’s website. Linking the two proprietary reservation systems presented an extreme challenge.
• Solution: The car rental company created a translation engine for sending data between the two systems.
04 October 2007 Kaiser: COMS W4156 Fall 2007 8
Example: Car Rental Company Interoperability
with Key Partner• Business Benefits: • Car rental company developed another large
sales channel
• Solution got to market quickly
04 October 2007 Kaiser: COMS W4156 Fall 2007 9
Tier 2 -- Interoperability with Key Partners
• The next step is to integrate one or two key partners outside the company
• Web Services allow for interoperability between applications across the public Internet
• But companies must agree upon the technologies they will use to develop these interoperating Web Services
04 October 2007 Kaiser: COMS W4156 Fall 2007 10
Example: Insurance Company Interoperability
across Several Companies• Background: A large insurer needed to generate
quotes for dental coverage and make them available on the intranet of one of their large corporate customers. But it had outsourced the maintenance of the dental providers directory and the credit rating service.
• Solution: The insurance company, credit rating service, and dental provider orchestrated these applications to generate a quote that was requested by the customer on a corporate intranet.
04 October 2007 Kaiser: COMS W4156 Fall 2007 11
Example: Insurance Company Interoperability
across Several Companies• Business Benefits: The insurance company
considered this a transformational competitive advantage for the following reasons:
• It generated quotes in half the time of its competitors and provided them via a corporate intranet to one of its major customers.
• It automated existing business relationships at the level of multiple, interoperating applications. As a result, outsourcing became much more valuable, cutting the cost of quote generation by one third.
04 October 2007 Kaiser: COMS W4156 Fall 2007 12
Tier 3 -- Interoperability across Multiple Companies
Companies want to extend their computing out to more partners and customers to build business ecosystems
04 October 2007 Kaiser: COMS W4156 Fall 2007 13
Business Ecosystem Requirements: Security
• The most common concern for companies implementing Web Services solutions
• Developers need an end-to-end security architecture that is straightforward to implement across companies and trust boundaries
04 October 2007 Kaiser: COMS W4156 Fall 2007 14
Business Ecosystem Requirements: Addressing
• Companies building Web Services solutions are concerned about the scalability and fault-tolerance of the business ecosystems they are building
• Developers need a way of specifying messaging paths and the ability to configure those message paths dynamically
04 October 2007 Kaiser: COMS W4156 Fall 2007 15
Business Ecosystem Requirements:
Reliable Messaging• A key requirement for mission-critical
applications
• Developers need an end-to-end guarantee of message delivery across a range of semantics such as: – at-least-once– at-most-once– exactly once
04 October 2007 Kaiser: COMS W4156 Fall 2007 16
Business Ecosystem Requirements: Coordination
• Some applications require database-like transactions across companies
• ACID properties – atomic, consistent, isolated, durable
• But because of the nature of decentralized computing, developers need flexible compensation-based transaction schemes
04 October 2007 Kaiser: COMS W4156 Fall 2007 17
Building Business Ecosystems
• The basic Web Services specifications enable interoperability between software components developed by different companies and residing on different infrastructures
• But most component model frameworks for use within an enterprise support security, reliability, transactions, etc.
• How can we add those capabilities to WS?
04 October 2007 Kaiser: COMS W4156 Fall 2007 18
Composable Services
• By adding more specialized Web Service specifications that are independent but can be combined
• For example, it is possible to independently add transaction identifiers and reliable messaging sequence numbers
• The two extensions do not conflict with each other and are compatible with pre-existing message structures
• Developers and providers can integrate selected specifications that fulfill the requirements of their communicating processes
04 October 2007 Kaiser: COMS W4156 Fall 2007 19
04 October 2007 Kaiser: COMS W4156 Fall 2007 20
SOAP Inherently Supports Composition
• SOAP uses a regular, multi-part message structure: New message elements supporting new services may be added to message headers in a manner that does not alter the processing of existing functionality
• SOAP body is for the ultimate recipient, SOAP header blocks may be targeted at any entity along the message path
04 October 2007 Kaiser: COMS W4156 Fall 2007 21
04 October 2007 Kaiser: COMS W4156 Fall 2007 22
04 October 2007 Kaiser: COMS W4156 Fall 2007 23
04 October 2007 Kaiser: COMS W4156 Fall 2007 24
Transports
• HTTP, HTTPS, SMTP (Simple Mail Transport Protocol)
• Core communication mechanisms • Move blocks of "bytes" between Web Services• This is only useful if participants can convert the
bytes into data structures that their code processes
04 October 2007 Kaiser: COMS W4156 Fall 2007 25
Messaging
• XML, SOAP• XML and XML Schema Definition (XSD) provide
the mechanisms for abstractly agreeing on message [data] structures
• SOAP defines the standard encoding for representing XML messages in the byte information that Web Services exchange over transports
04 October 2007 Kaiser: COMS W4156 Fall 2007 26
Addressing
• Messages and responses both go somewhere and come from somewhere (and errors also need to be reported somewhere)
• By default, SOAP encodes the destination for a message with a URL placed in the HTTP transport
• The destination for the response is determined by the HTTP return address
• Builds on the basic browser-server model
04 October 2007 Kaiser: COMS W4156 Fall 2007 27
Addressing
• The source and destination information are not part of the message itself
• But information can be lost if a transport connection terminates (e.g., if the response takes a long time and the connection times out)
• Or if the message is forwarded by an intermediary, perhaps routed over multiple transports
• Also does not allow for directing a response to a third party (e.g., request sent over HTTP but returned via SMTP)
04 October 2007 Kaiser: COMS W4156 Fall 2007 28
WS-Addressing• Provides a mechanism to place the target, source
and other addressing information directly within the message
• Decouples address information from any specific transport model
• Supports both asynchronous and extended duration communication patterns
• But not very exciting for request/response over a single HTTP connection (see blog entry)
04 October 2007 Kaiser: COMS W4156 Fall 2007 29
Message Addressing Properties• To -- message destination, required (URI)• Action -- an action value indicating the semantics of the
message, corresponds to WSDL porttype, required (URI)
• From -- the endpoint of the service that dispatched this message (EPR)
• ReplyTo -- the endpoint to which reply messages should be dispatched (EPR)
• FaultTo -- the endpoint to which fault messages should be dispatched (EPR)
• Unique MessageId, required if there will be any response (URI)
• RelatesTo previous messages (a pair of URIs, indicating previous From and MessageId)
04 October 2007 Kaiser: COMS W4156 Fall 2007 30
Example with Simple Endpoint References
<S:Envelope xmlns:S="http://www.w3.org/2003/05/soap-envelope" xmlns:wsa="http://www.w3.org/2004/12/addressing"> <S:Header> <wsa:MessageID> http://example.com/SomeUniqueMessageIdString </wsa:MessageID> <wsa:ReplyTo>
<wsa:Address>http://myClient.example.com/someClientUser</wsa:Address> </wsa:ReplyTo> <wsa:FaultTo>
<wsa:Address>http://myserver.example.com/DemoErrorHandler</wsa:Address> </wsa:FaultTo> <wsa:To>mailto:[email protected]</wsa:To> <wsa:Action>http://myserver.example.com/DoSomething</wsa:Action> </S:Header> <S:Body> … </S:Body></S:Envelope>
04 October 2007 Kaiser: COMS W4156 Fall 2007 31
WS-Addressing Endpoint Reference
• A standard XML element providing a structured approach to encoding fine-grained addressing
• Only the address is required• May include reference properties that distinguish
between multiple services (or multiple versions of the same service) at the same address
• May include reference parameters that identify resources managed by a particular service
04 October 2007 Kaiser: COMS W4156 Fall 2007 32
Example Extended Endpoint Reference
<wsa:EndpointReference xmlns:wsa="..." xmlns:example="...">
<wsa:Address>http://example.com/weather</wsa:Address>
<wsa:ReferenceProperties>
<example:ServiceLevel>Basic</example:ServiceLevel> </wsa:ReferenceProperties> <wsa:ReferenceParameters> <example:CityCode>NYC</example:CityCode> </wsa:ReferenceParameters></wsa:EndpointReference>
04 October 2007 Kaiser: COMS W4156 Fall 2007 33
04 October 2007 Kaiser: COMS W4156 Fall 2007 34
Description
• The transport and message specs allow Web services to communicate using messages
• But how do participants know what the messages are?
• How does a Web Service describe the messages it sends and receives?
• Using a Web Service requires an understanding of the messages the Web Service will consume and produce
04 October 2007 Kaiser: COMS W4156 Fall 2007 35
XSD and WSDL
• XML Schema allows developers and service providers to define XML types for data structures, e.g., a purchase order, and messages, e.g., the CreatePO message
• WSDL allows a Web Service to document the messages it receives and sends - what "actions" or "functions" the service performs in terms of the messages it receives and sends
04 October 2007 Kaiser: COMS W4156 Fall 2007 36
Requirements and Expectations between
Requesters and Receivers• WSDL and XSD define the service's interface
syntax, but do not express information about what the service requires/expects of the caller and vice versa
• For example, does the service require security or implement transactions?
• WS-Policy enables a service to specify the functional assurances that they expect from and provide to callers (and intermediaries)
04 October 2007 Kaiser: COMS W4156 Fall 2007 37
WS-Policy• Extensible language for expressing the
requirements, capabilities and preferences of a service
• Assertions represent an individual preference, requirement or capability
• Usage: Required, Optional, Rejected, Observed, Ignored
• Operators: All, ExactlyOne, OneOrMore• Applied to domain-specific policy subjects (e.g.,
from WS-SecurityPolicy)
04 October 2007 Kaiser: COMS W4156 Fall 2007 38
Example<wsp:Policy xmlns:wsse="..." xmlns:wsp="...">
<wsp:ExactlyOne> <wsse:SecurityToken wsp:Usage="wsp:Required"
wsp:Preference="100"> <wsse:TokenType>wsse:Kerberosv5TGT</wsse:TokenType> </wsse:SecurityToken> <wsse:SecurityToken wsp:Usage="wsp:Required"
wsp:Preference="1"> <wsse:TokenType>wsse:X509v3</wsse:TokenType>
</wsse:SecurityToken></wsp:ExactlyOne>
</wsp:Policy>
04 October 2007 Kaiser: COMS W4156 Fall 2007 39
Obtaining Descriptions
• XML, XSD, WSDL and WS-Policy support describing the interface and service assurances for a Web Service
• But how do potential users of the service find this information?
• Currently the most common approach is through email exchanges, manual (human) online lookup, or word of mouth
• A more general purpose, scalable model is necessary – although has not (yet) proven practical outside an intranet setting
04 October 2007 Kaiser: COMS W4156 Fall 2007 40
UDDI
• UDDI = Universal Description and Discovery Interface
• Query UDDI at design time, to find services compatible with requirements
• Query UDDI at runtime, when the caller "knows" the interface it requires and searches for a service that meets its functional requirements or is provided by a well-known partner
04 October 2007 Kaiser: COMS W4156 Fall 2007 41
WS-MetadataExchange
• The caller service may go directly to the callee service to obtain information via SOAP messages following WS-MetadataExchange specification
• Used when developers already have a reference to a service and need to obtain details
• Obtains relevant WSDL, XSD, WS-Policy, etc.
04 October 2007 Kaiser: COMS W4156 Fall 2007 42
04 October 2007 Kaiser: COMS W4156 Fall 2007 43
Requirements and Expectations Again (Assurances)
• It is not enough to simply exchange messages• Applications and services reside in middleware
and systems that provide valuable component services such as security, reliable messaging and transacted operations
• Web Services need a mechanism for interoperability between these facilities
04 October 2007 Kaiser: COMS W4156 Fall 2007 44
Security
• Support authentication, message integrity, confidentiality, trust, privacy
• Federation of security between different organizations
04 October 2007 Kaiser: COMS W4156 Fall 2007 45
Security Requirements
• A sends a message to service B• B partially processes the message and forwards
it to service C• HTTPS allows authentication, integrity, and
confidentiality between A-B and B-C• However, C and A cannot authenticate each
other, or hide information from B • For A, B and C to use BASIC-Auth for
authentication, they must share the same replicated user and password information
04 October 2007 Kaiser: COMS W4156 Fall 2007 46
WS-Security
• Defines mechanisms for associating security related claims with a message
• Signed, encrypted security tokens– Username/password– x509 certificates– Kerberos tickets– XML-based tokens: XrML eXtensible rights Markup
Language) and SAML (Security Assertion Markup Language)
04 October 2007 Kaiser: COMS W4156 Fall 2007 47
WS-Security
• A can generate a token that C can verify as having come from A, B cannot forge the token
• A can sign selected elements or the entire message, this allows B and C to confirm that the message has not changed since A sent it
• A can seal the message or selected elements, this ensures that only the intended service for those elements can use the information and his prevents B from seeing information intended for C and vice versa
04 October 2007 Kaiser: COMS W4156 Fall 2007 48
Trust
• Security relies on pre-defined trust relationships• Kerberos works because participants "trust" the
Kerberos Key Distribution Center• PKI (public key infrastructure) works because
participants trust the root certificate authorities
04 October 2007 Kaiser: COMS W4156 Fall 2007 49
WS-Trust
• Defines an extensible model for setting up and verifying trust relationships
• Allows Web Services to set up and agree on which security servers they "trust" and to rely on these servers
• The key concept is a Security Token Service (STS) - a distinguished Web Service that issues, validates and exchanges security tokens
• An STS might issue a Kerberos token asserting that the key holder is Susan, based on an X.509 certificate issued by a trusted Certificate Authority
• Enables organizations using different security technologies to federate
04 October 2007 Kaiser: COMS W4156 Fall 2007 50
STS
04 October 2007 Kaiser: COMS W4156 Fall 2007 51
Secure Conversations
• Some Web Service scenarios only involve the short sporadic exchange of a few messages – which is what WS-Security was intended for
• Other scenarios involve long duration, multi-message conversations between the Web Services
• WS-Security may not be good enough for:– Repeated use of computationally expensive
cryptographic operations such as public key validation– Sending and receiving many messages using the
same cryptographic keys, providing more information that allows brute force attacks to "break the code"
04 October 2007 Kaiser: COMS W4156 Fall 2007 52
Secure Conversations
• Protocols like HTTPS use public keys to perform a simple negotiation that defines conversation specific keys
• This key exchange allows more efficient security implementations and also decreases the amount of information encrypted with a specific set of keys
04 October 2007 Kaiser: COMS W4156 Fall 2007 53
WS-SecureConversation
• WS-SecureConversation provides similar support to WS-Security
• Participants may use WS-Security with public keys to start a "conversation" or "session“, and use WS-SecureConversation to agree on session specific keys for signing and encrypting information
04 October 2007 Kaiser: COMS W4156 Fall 2007 54
Federation
• Sometimes a set of organizations need to establish a single virtual security domain
• For example, a travel agent, an airline and a hotel chain may set up such a federation
• An end-user that "logs into" any member of the federation has effectively logged into all of the members
04 October 2007 Kaiser: COMS W4156 Fall 2007 55
WS-Federation
• WS-Federation defines several models for providing federated security through WS-Trust and WS-SecureConversation
• Customers often have "properties" when they deal with an enterprise
• An example is a preference for window or aisle seats, or a midsize car
• WS-Federation allows the members to set up a federated property space
04 October 2007 Kaiser: COMS W4156 Fall 2007 56
WS-Federation
• Properties about individuals may be closely held for privacy protection or because the information provides a competitive advantage to a specific federation member
• WS-Federation supports a pseudonym model• For example, users that have authenticated to
the travel agency have agency-generated "aliases" in their interactions with the airline or hotel
04 October 2007 Kaiser: COMS W4156 Fall 2007 57
04 October 2007 Kaiser: COMS W4156 Fall 2007 58
Reliable Messaging
• In an Internet world, almost all communication channels are unreliable - messages disappear or are duplicated, connections break
• Without a reliable messaging standard, Web Service application developers must build these functions into their applications
• The basic approaches and techniques are well understood, e.g., many middleware systems ensure messages have unique identifiers, provide sequence numbers, and use retransmission when messages are lost
• If Web Service developers implement these models in their applications, they may make different assumptions or design choices, resulting in little if any reliable messaging
04 October 2007 Kaiser: COMS W4156 Fall 2007 59
WS-ReliableMessaging
• Defines mechanisms that enable Web Services to ensure delivery of messages over unreliable communication networks
• Supports bridging two different infrastructures into a single, logically complete, end-to-end model
04 October 2007 Kaiser: COMS W4156 Fall 2007 60
WS-ReliableMessaging
• The RM Source MUST assign each reliable message a sequence number beginning at 1 and increasing by exactly 1 for each subsequent reliable message
• Every acknowledgement issued by the RM Destination MUST include within an acknowledgement range or ranges the sequence number of every message successfully received by the RM Destination and MUST exclude sequence numbers of any messages not yet received
04 October 2007 Kaiser: COMS W4156 Fall 2007 61
WS-ReliableMessaging
• Delivery Assurances – AtMostOnce, AtLeastOnce, ExactlyOnce, InOrder
• Protocol Elements – Sequence, Sequence Acknowledgement, Request Acknowledgement, Sequence Creation, Sequence Termination
• Policy Assertions – SequenceCreation, SequenceExpiration, InactivityTimeout, RetransmissionInterval, AcknowledgementInterval
04 October 2007 Kaiser: COMS W4156 Fall 2007 62
04 October 2007 Kaiser: COMS W4156 Fall 2007 63
04 October 2007 Kaiser: COMS W4156 Fall 2007 64
Transactions• A complex business scenario may require
multiple parties to exchange multiple sets of messages
• The multiple messages exchanged between participants constitute a logical "task" or "objective"
• The parties must be able to: – Start new coordinated tasks. – Associate operations with their logical task - the
parties may be performing multiple such tasks at the same time
– Agree on the outcome of the computation
04 October 2007 Kaiser: COMS W4156 Fall 2007 65
WS-Coordination
• General mechanism for starting and agreeing on the outcome of multiparty, multi-message Web service tasks
• Coordination context is a message element that flows on all messages that Web Services exchange during the computation
• The coordination context contains the WS-Addressing endpoint reference to the coordination service and the endpoint contains information to identify the specific task being coordinated
04 October 2007 Kaiser: COMS W4156 Fall 2007 66
Coordination Service
• Starts a coordinated task, terminates a coordinated task, allows a participant to register in a task, and produces a coordination context that is part of all messages within a group
• Includes an interface that participating services use in order to be informed of the outcome of the coordinated task
04 October 2007 Kaiser: COMS W4156 Fall 2007 67
04 October 2007 Kaiser: COMS W4156 Fall 2007 68
WS-AtomicTransaction
• Defines a specific set of protocols that plug into WS-Coordination to implement traditional two-phase atomic transactions
• For activities that require the traditional atomic, consistent, isolated, and durable (ACID) properties
• Usually short-lived
04 October 2007 Kaiser: COMS W4156 Fall 2007 69
WS-AtomicTransaction
• Atomic, two-phase model is only with respect to the services involved
• Sites or infrastructure offering services may advertise two-phase commit, but use some other intra-enterprise model like compensation or versioning
• This makes a simple two-phase commit model more useful for long-running Internet computations
04 October 2007 Kaiser: COMS W4156 Fall 2007 70
Business Activities
• May consume many resources over a long duration
• May involve a significant number of atomic transactions
• Individual tasks within a business activity can be “seen” prior to the completion of the business activity - their results may have an impact outside of the computer system
• Responding to a request may take a very long time - human approval, assembly, manufacturing or delivery may have to take place before a response can be sent
04 October 2007 Kaiser: COMS W4156 Fall 2007 71
Business Activities
• In the case where a business exception requires an activity to be logically undone, abort is typically impractical/impossible
• Exception handling mechanisms may require business logic, e.g., in the form of a compensation task, to reverse the effects of a completed business task
04 October 2007 Kaiser: COMS W4156 Fall 2007 72
WS-BusinessActivity• Another set of protocols that plug into WS-
Coordination, to coordinate activities that apply business logic to handle business exceptions
• Actions are applied immediately and are permanent
• Compensating actions may be invoked in the event of an error
• Enables existing business process and work flow systems to wrap their proprietary mechanisms and interoperate across trust boundaries and different vendor implementations
04 October 2007 Kaiser: COMS W4156 Fall 2007 73
04 October 2007 Kaiser: COMS W4156 Fall 2007 74
BPEL4WS = Business Process Execution Language for Web Services
• Supports service composition• Enables developers to define the structure and
behavior of a set of Web Services that together implement a shared business solution
• The composed solution is itself a Web Service, which supports HTTP/SOAP messages and defines its interface using WSDL (and WS-Policy)
04 October 2007 Kaiser: COMS W4156 Fall 2007 75
BPEL4WS Processes
• Executable processes model actual behavior of each participant in a business interaction
• Abstract processes specify the mutually visible message exchange behavior of each of the parties involved in the protocol, without revealing their internal behavior
• And much much more…• Basically an entire programming system where
conventional Web Services are the atomic functions
04 October 2007 Kaiser: COMS W4156 Fall 2007 76
04 October 2007 Kaiser: COMS W4156 Fall 2007 77
Revised Project ConceptDue Next Week!
• Tuesday 9 October, 10am
• Assignments posted on course website
• Submit via CourseWorks
• Revised Project Concept
04 October 2007 Kaiser: COMS W4156 Fall 2007 78
Upcoming Deadlines
• First iteration plan due October 16th • First iteration progress report due October 23rd • First iteration demo week October 30th –
November 8th • First iteration final report due November 9th
04 October 2007 Kaiser: COMS W4156 Fall 2007 79
COMS W4156: Advanced Software Engineering
Prof. Gail Kaiser
http://york.cs.columbia.edu/classes/cs4156/