The Web: Distributed Objects Realized!

Mark Baker, Coactus ConsultingStuart Charlton, BEA Systems

2

About Us

Mark BakerCoactus Consulting

CTO-level consulting and large-scale distributed systems architect

Leading proponent of “REST style web services” for over seven years

Prior work at Nortel, Sun Microsystems, and several startups

Co-developed internet, mobile, and web standards through the IETF, W3C, and OMA

Mark’s Bloghttp://www.markbaker.ca/blog/

mailto:mark@coactus.com

Stuart CharltonBEA Systems

Enterprise architect in BEA’s Consulting practice

Co-develops BEA’s global consulting strategy and service offerings

Specialized in distributed systems architecture, data warehousing, and object technology

Prior work in EA, consulting, and mentoring both the financial services & telecommunications industries

Stu Says Stuffhttp://stucharlton.com/blog/

mailto:stuart.charlton@bea.com

3

Agenda

Part 1

How We Got Here

Understanding Distributed Object Interoperability

Exploring the Design of REST Identity and Reference Handling Evolution Multi-Organizational

Interoperability Control Flow through Hypermedia

Part 2

RESTful Web Services in Action

Exploring the Web’s Architecture

Applied Techniques for RESTful Web Services Resources-Oriented Architecture Security Approaches Transactions & Reliability

How We Got Here

The Rise of the Object Web

5

Where we’ve come from…

“The next shift catalyzed by the Web will be the adoption of enterprise systems based on distributed objects and IIOP (Internet Inter-ORB Protocol).

IIOP will manage the communication between the object components that power the system. Users will be pointing and clicking at objects available on IIOP-enabled servers.”

- Marc Andressen,Netscape Cofounder(October, 1996)

6

The Object Web Shall Arise!

“To move to the next step, the Web needs distributed objects. We call this next wave of Internet innovation the "Object Web.“

The HTTP form you submit is still the basic unit of client/server interaction. This clumsy work-around is not suitable for full-blown client/server applications that require highly interactive conversations between components. It also does not scale well.”

- CORBA, Java, and the Object Web BYTE Magazine (October 1997)

7

8

By 1999…. XML Web Services Shall Arise!

…Over the past decade, component technology has been the cause of many arguments, disagreements, and debates.

This component-induced friction can be traced to two primary factors:

1. “My Glue is Better than Your Glue”

2. “I Can’t Believe You Program In That Language”

- Don BoxLessons from the Component Wars: An XML Manifesto(September 1999)

9

By the mid-2000s…. what about “just” the Web?

10

Our Claims:

1. The dream of the “Object Web” is largely here: it’s the Web!

2. Constraints in a systems architecture lead to emergent properties.

3. The Representational State Transfer (REST) architectural style defines constraints which helped guide the web towards a globally scalable, interoperable, and extensible system.

4. The REST architectural style is both an evolution of past approaches to networked computing, and a radical departure.

5. More work remains to be done!

Understanding Distributed Object Interoperability

12

The Distributed Object Request Broker (ORB)

The Goals of an ORB:

Encapsulate all of the mechanisms to:Find the object for the request

Prepare the implementation to receive the request

Communicate the data making up the request

Ensure complete transparency of object location and implementation

13

Inside an Object Request Broker

Source: OMG CORBA 2.3 Architecture & Specification

14

How can a web of distributed objects span organizations?

15

Interoperability Domains

A distributed system, even if the goal is transparency, has several co-existing domains of interoperability

A domain is a set of objects sharing a common characteristic or abiding by common rules.Source: OMG CORBA 2.3 Architecture & Specification

16

Types of Interoperability Domains

Referencing The scope of a distributed object reference

Representation The scope of a message transfer syntax & protocol

Network Connectivity Potential scope of a network message

Network Addressing Potential scope of a network address

Security The extent of a particular security policy

Others…

Source: OMG CORBA 2.3 Architecture & Specification

17

Solutions to Multiple Interoperability Domains

Canonicalization

Simple and scalable, at the cost of reduced runtime performance

Low barrier to entry for new clients, high barrier for new servers

Point-to-Point Adaptation

Good runtime performance,not as scalable, increasing complexity

Low barrier to entry for new servers, high entry for new clients

CanonicalDomain

CanonicalDomain

Domain ADomain A Domain BDomain B

Domain ADomain A

Domain CDomain C Domain BDomain B

18

Solutions for a “Global Object Web”?

Goal seems to be primarily about use and reuse: more consumption Thus, lower the barriers to new clients!

Scope of Object References: Global

Scope of Representation: Global (but extensible)

Scope of Network Connectivity: Global

Scope of Security: Shared Agreement Requires at least some level of Global Canonicalization

19

Solutions for a “Global Object Web”?

Further Concerns:

Unpredictable type, usage, and volume of information Thus, shaped around extensible, streamable, cacheable, identifiable

information

Both Human and Automated actors Thus, simple, reusable, and extensible interfaces, appropriate for either

… How would you architect such a system? This is atypical of most distributed object systems architectures in practice

20

Two Very Different Types of Architectural Domains

Client or Builder-focused

Identifiable Client or Customer(s) Clearly identifiable builders Clearly identifiable users Control governance & control Overall architectural control System is a result of deliberate

value judgement by the client

Examples Most IT projects, software

products, civil structures, etc.

Collaborative Systems

No identifiable client Many builders Many (unpredictable) users No central control in terms of

development, or operation System is a result of the voluntary

choices of the participants

Examples Internet and WWW, electrical

power systems, joint military operations, etc.

Source: Eberherdt Rechtin, The Art of Systems Architecting

21

Building Emergent Capability into an Architecture

Collaborative systems are about creating emergent capability

The architecture of the system is the constraints on interfaces Components & implementations are out of scope and out of control

An example style of such constraints: Client/Server Architecture

Constraints:

Active client, Reactive server, client initiates communication Emergent properties:

Simplifies server, enables multiple organizational domains

Client Server

22

Composing Constraints into an Architectural Style

Constraints both enable and detract from emergent properties, and may be combined together

Example: Client/Server + Layered System + Caching

+ Stateless Interaction

Enables: Simplified Clients and Servers (due to statelessness & C/S) Shared Caching (due to layering) Improved Visibility & Reliability (due to statelessness)

Add in: Code on Demand (COD)

Result: Simplifies clients, Improves Extensibility, but Reduces Visibility

Source: Roy Fielding, Architectural Styles and the Design of Network-Based Software Architectures

23

The Representational State Transfer (REST) Style

Originally referred to as “The HTTP Object Model”

Source: Roy Fielding, Architectural Styles and the Design of Network-Based Software Architectures

24

Some Major Differences between REST and Distributed Objects

1. Most distributed architectures do not assume application requirements Yet, the Web is an application with goals & requirements! The overall goal: sharing information and capabilities at a global scale REST assumes the requirement is to provide an architecture of

participation for a globally networked information space

2. Most distributed architectures do not require globally scoped interoperability domains REST insists on a uniform interface to ensure global scope interoperability The uniform interface is what we explore next!

These differences apply elsewhere:SOAP Web Services, RPC, etc. are also just general toolkits

25

RESTful Distributed Objects in contrast to SOAP Web Services

Perceived advantages:

Communication occurs through XML document exchange Agreement occurs in the network protocol

The XML InfoSet is the basis of interoperability No worrying about object-oriented purity / language-oriented bindings

Disadvantages: Prefers interface descriptions for specific (non-uniform) interfaces All data must flow through the XML InfoSet, other types are secondary Nearly eliminates the notions of:

Object lifecycle (creation, removal, activation)

Object identification & references (i.e. linking) Assumes transfer protocol independence is possible and desirable

In practice, transfer protocol semantics leak into message exchanges

26

A Word on REST vs. The World Wide Web

Consider three levels of abstraction:1. Architectural Styles: Representation State Transfer (REST)

2. Architecture: The World Wide Web

3. Implementation: “User Agents, Web Servers, Proxies, etc.”

REST is a model of constraints and emergent properties

The Web is the most widely used and understood implementation

Exploring the Design of REST

28A Global, Networked, Shared Information Space

29Users Consume & Manipulate Information via Agents

30Information Objects Must Be Identifiable

31An Object’s Interface is separated from Implementation

32State is exchanged in a stateless, self-describing manner

33

Handling Evolution

34Meaning is Determined by the Context of Relationships

35Content & Relationships May Change Over Time...

36...But the Identity of Information Must Remain Stable

37

The Information Space consists of Identifiable Conceptual Mappings

State(at a point in time)

Object

38

Instead of Objects & State, REST calls these Resources & Representations

Representation(at a point in time)

Resource

39

The conceptual mapping should remain consistent over time!

Concept

Representation(at a point in time)

Resource

“Customer 31319”

Concept

“The state of Customer 31319 at a

point in time”

Server

Client

(stable)

40

Example:Different Representations

Concept

XML Document

Resource

“Customer 31319”

Concept

“The state of Customer 31319 at a

point in time”

Server

Client

(stable)

CSV File

41

Multi-Organization Interoperability

42An Organization Manages Information

43Information Spans Organization Boundaries

44Clients Span Organizational Boundaries

45Clients & Infrastructure Evolve Independently

46

So, representations must be uniformly specified

47

What Domains are in a Representation?

The means and intent of communication All clients and servers should at least be able to

communicate, regardless of the particular resource Thus, communication must be very generic and

uniformly applicable to all resources

The information bits themselves, along with their self-descriptive interpretation Information can be represented in many ways, even if

it’s the same concept

The references for the requested or related resources Relationships between a medium identifiers can

denote the meaning of information

48

How Stable Should the Domains of a Representation Be?

Data ElementData Element Level of StabilityLevel of Stability Authority over ChangeAuthority over Change

Means & Intent of Communication

Extremely Stable Global

Representation of InformationVery Stable Widely Shared Agreement

Information Resource IdentifierStable Organization

49

Control Flow

50

Embed Control FlowInside the Representation Itself

51

As Information is Manipulated or Conveyed,So Are The References for “Next Steps”

52

Hyperlinks Define thePossibilities for Communication

53Application State is Held By the Agent

54

Summary of the REST Uniform Interface

All valuable resources are identified by a globally scoped, uniform, resource identifier Simple, consistent ability to (de)reference both information & services

Resources are manipulated through the exchange of representations Stateless, visible communication to encapsulated resources

All exchanges are self-descriptive Enables visibility and extensibility

All access methods mean the same for all resources Communication semantics are universal

Hypermedia is the engine of application state Information and controls are intertwingled

55

Agenda

Part 1

How We Got Here

Understanding Distributed Object Interoperability

Exploring the Design of REST Identity and Reference Handling Evolution Multi-Organizational

Interoperability Control Flow through Hypermedia

Part 2

RESTful Web Services in Action

Exploring the Web’s Architecture

Applied Techniques for RESTful Web Services Resources-Oriented Architecture Security Approaches Transactions & Reliability

RESTful Web Servicesin Action

57

RESTful Light Bulbs

Imagine any lightbulb, anywhere, is an HTTP-supported resource

Multiple representations might be supported Application/lightbulb+xml An HTML form

E.g. Turning on a lightbulb:PUT http://example.com/kitchen/main HTTP/1.1

Content-Type: application/lightbulb+xml

<lightbulb>

<state>on</state>

</lightbulb>

How does this impact the scale of our lightbulb system?

58

A Hypermedia Purchase Order

Without URIs and Links

POST /purchaseOrders HTTP/1.1

<?xml version="1.0"?>

<updatePurchaseOrder>

<purchaseOrder id=“31811” orderDate="1999-10-20">

<shipTo country="US">

<name>Alice Smith</name>

<street>123 Maple Street</street>

<city>Mill Valley</city>

<state>CA</state>

<zip>90952</zip>

</shipTo>

<items>

<item partNum="872-AA"> <quantity>1</quantity>

<USPrice>148.95</USPrice>

<comment>Confirm this is electric

</comment>

</item>

</items>

</purchaseOrder>

</updatePurchaseOrder>

With URIs Hyperlinks

PUT /purchaseOrder/31811 HTTP/1.1

<?xml version="1.0"?>

<purchaseOrder orderDate="1999-10-20">

<shipTo

href=“http://alice.com/address”/>

<items>

<item href=“http://lawnsrus.com/872-AA”>

<quantity>1</quantity>

<comment>Confirm this is electric

</comment>

</item>

</items>

</purchaseOrder>

How do either of these impact evolution over time?

59

Exposing Legacy Information as Resources

Reusing web Infrastructure to support the legacy

E.g. Cacheable, Searchable, Composable Customer Data

RESTful Web Service Gateway

Legacy Customer Information System

Composite App

DB

Web Cache Web Crawler

http://company.com/customer/30133

Other

Resources

Exploring the Web Architecture

Highlights

61BEA Confidential | 61

Web Architecture

A View of the Current Web Architecture

Foundation

Internet ProtocolFoundation

Hypertext Transfer Protocol (HTTP)

Sec

uri

ty &

Au

then

tica

tio

n

Man

agem

ent

& C

ach

ing

BGP

ApplicationProtocolStandards

ARPDNSIPTCP

Multipurpose Internet Mail Extensions (MIME)

Interoperability

CrawlersCrawlersUser AgentsUser Agents Semantic WebSemantic Web MicroformatsMicroformats

Web Sites Resource-OrientedSystem Interfaces

ContentSyndication

Media Types

HTMLHTML XMLXML RSS / AtomRSS / Atom

URI/IRIUnicode

JavaScriptJavaScript ImagesImages VideoVideo

JSONJSON

Form dataForm data

62

Inside an Object Request Broker

Source: OMG CORBA 2.3 Architecture & Specification

63

The New ORB: A Web Server

64

Hypertext Transfer Protocol

GET /docs/2.2/ HTTP/1.1\r\nHost: httpd.apache.org\r\nAccept: text/html, text/*, */*\r\nUser-Agent: GET/7 libwww-perl/5.40\r\n\r\n

HTTP/1.1 200 OK\r\nDate: Thu, 31 Jul 2007 15:40:09 GMT\r\nServer: Apache/2.2.4\r\nContent-Type: text/html\r\nContent-Language: en\r\nTransfer-Encoding: chunked\r\nEtag: “b381faa81c”\r\nCache-control: max-age=3600\r\nVary: Accept-Language\r\n\r\n 4090\r\n<HTML><HEAD> …

65

Highlights of the HTTP Interface

Methods

GET Retrieve a resource’s state

PUT Store/replace a resource’s state

POST Create/append to a resource’s state

DELETE Remove a resource’s state

HEAD Retrieve resource & control metadata

OPTIONS Retrieve what methods are supported

TRACE Retrieve a loop-back of the request

CONNECT Request that a proxy tunnel the request

Response Code Ranges

100…Informational

200…Success indicators Request has succeeded New resource created

300…Redirection Content negotiation Resources that have moved permanently or

temporarily Cacheable resources that have not changed

400…Bad Requests Method not supported Unauthorized access Resource not found currently or permanently

500…Server Errors Feature not implemented Service temporarily unavailable Internal server errors

66

How Stable Should a Web Representation Be?

Data ElementData Element Level of StabilityLevel of Stability Authority over ChangeAuthority over Change

Means & Intent of Communication

(e.g. identifiers, operations & metadata)

Extremely StableGlobal

(HTTP & URI Syntax)

Representation of InformationVery Stable

Widely Shared Agreement(MIME Types)

Information Resource IdentifierStable

Organization(Allocated URIs)

67

How does a Client Understand its Information?

Stable, standardized, streaming media formats Multipurpose Internet Mail Extensions (MIME) Types Generic containers for more specific information

The context of relationships to other identifiers Information may embed links within a document E.g. an HTML tag <IMG SRC=“http://images/customer/31319...”> implies:

A link is to an associated resource

This resource should respond to HTTP GET

This resource will be an image of some type

Code-on-demand that can interpret the information for the client E.g. Java Applets, ActiveX Controls, Flash Plug-ins, JavaScript, AJAX

68

Intermediaries

Proxies are client-determined intermediaries

Gateways are server-determined intermediaries

Caching with HTTP Conditional GET

GET /docs/2.2/ HTTP/1.1\r\n

Host: httpd.apache.org\r\n

Accept: text/html, text/*, */*\r\n

If-None-Match: “38d8af8210bb”\r\n

\r\n

HTTP/1.1 304 Not Modified\r\n

Content-Type: text/html\r\n

Content-Length: 0\r\n

ETag: “38d8af8210bb”\r\n\r\n

ProxyUser Agent

(Browser)

GatewayServer

69

Some flaws in the Deployed Web Architecture

MIME Types Benefit: reuse most of the metadata

and media descriptions for email But, designed for Email, not the web Focus on packaging lots of

information into a single package

Hypermedia prefers linking to subsets

Workarounds for layered content

e.g. GZIP transfer encoding of a text/html document

Assumes a lossy transport Centralized type registry

Cookies Site-wide opaque data

Reduced visibility, privacy, and security

Can lead to counter-intuitive behaviour

Some sites do not handle the browser’s back button

Persistent data should be a resource

Eg. A Shopping Cart URI

But, requires browser extensibility

Alternatives are emerging with the popularity of AJAX-like techniques to store & retrieve preferences

Applied Techniques for RESTful Web Services

71

A Design Method forResource-Oriented Architecture

1. Figure out the data set

2. Split the data set into resources

3. For each resource:a. Name the resources with URIs

b. Expose a subset of the HTTP interface

c. Design the representation(s) accepted from the client

d. Design the representation(s) served to the client

e. Integrate this resource to others, using hypermedia links and forms

4. Consider use casesApply standard hypermedia types (e.g. HTML, Atom), where applicable

5. Consider error conditionsApply standard hypermedia types (e.g. HTML, Atom), where applicable

72

Common & Emerging Hypermedia Types

The big six:

XHTML, HTML, Flash, PDF, RSS, and Atom

Emerging:

Atom Publishing Protocol, HTML Microformats

On the horizon:

HTML 5

RDF, GRDDL and Semantic Web standards

Media types usually define a general purpose control flow:

73

RESTful Security Approaches

Identity Assertion OpenID

URI-based digital identity

Open ID 2.0 is adding attributes, secure messages

Kerberos / SPNEGO

Popular on intranets, requires a server

Not likely internet scalable

Current implementations require excessive round-trips

74

RESTful Message Confidentiality & Integrity

Mutual SSL Robust, widely supported Does not enable intermediary visibility or

persistent signatures

XML Signature & Encryption But XML Canonicalization is extremely

complicated

S/MIME or PGP Key negotiation is usually out of band

OpenID 2.0 may help Widely deployed within the email system

(perhaps not as widely used) Fits within existing HTTP infrastructure Not widely deployed within HTTP Limitations:

http://blog.jclark.com/2007/10/why-not-smime.html

Image: Mark Wahl, ldap.com

75

Transactions & Lost Updates

Conditional PUT A RESTful approach to optimistic locking

PUT /test_resource HTTP/1.1\r\nHost: bea.com\r\nAccept: */*\r\nExpect: 100-continue\r\nIf-None-Match: “381fa9311bc”\r\n\r\n

HTTP/1.1 412 Precondition Failed\r\nDate: Thu, 31 Jul 2007 15:40:09 GMT\r\nServer: Apache/2.2.4\r\nContent-Type: text/html\r\n

76

RESTful Distributed TransactionsDesign Sketch: Resource-Oriented Units of Work Resource for the Transaction Manager & each Transaction A new resource is created for each registered resource, enabling isolation Transaction manager acts as an agent to register resources, prepare & commit

77

Reliable Messaging

Ordered messaging Assumes a protocol will order stateful, asynchronous application exchange

patterns Whereas, REST assumes stateless, synchronous application exchange

patterns

Each request requires a response

Requests may be pipelined, will be responded to in-order

Exactly-once processing Legitimate need for resource state changes, e.g. In HTML forms, eliminating the “double form submit” button Two possible solutions:

Application-level check (e.g. unique ID in HTML hidden form field)

POST Exactly Once: http://www.mnot.net/drafts/draft-nottingham-http-poe-00.txt

Summary

79

Summary

The “Global Object Web” is largely here: it’s the Web!

The Web is similar to a Distributed Object System Objects ~= Resources Resources may be referenced, created, updated and/or deleted

Global-scale interoperability requires constraints on communication Unpredictable evolution & configurations are the norm Serendipitous reuse is more common than planned reuse Must depend on a stable, but extensible, uniform interface to communicate

Many traditional distributed systems challenges can be solved in a RESTful manner

Reliability, optimistic locking, security, transactions

More work remains to be done – “Learn to love the Web”!

Thank you!

Alternative Architectures

When is REST not appropriate?

82

Alternative Architectures

Transparent Remote Access Developer transparency is paramount Information sharing is unimportant Evolution is federally controlled (e.g. producers & consumers are under

the same control ) Distributed Objects or XML Web Services can be appropriate

Globally Consistent Transaction System Tightly Coupled Single-System Image (e.g. a cluster) Consistency trumps High Availability Accomplished through language-level bindings (e.g. Java XA, MSDTC)

or distributed object network protocols (e.g. TIP or IIOP)

83

Alternative Architectures

Stateful, In-Order, Reliable Messaging When you don’t have the time or inclination to share data Most interactions are not safe or idempotent The application does not lend itself well to statelessness The physical scale (number of servers) is relatively small at each layer Clients are trusted & controlled in the same federation as the servers The network is controlled & predictable

Structured Data Analysis, Manipulation and Reporting Remote Data Access (SQL, SPARQL) interfaces use general-purpose

connectors Requires extensive knowledge of the schema Often requires per-client state maintenance

84

Alternative Architectures

Stateful, real-time two-way communication E.g. voice or video communication / conferencing, P2P file transfer Requires a synchronous two-way streaming architecture

However, stream signaling and negotiation arguably can be RESTful

E.g. BitTorrent registries & trackers require the Web to function

85

Alternative Architectures

High-speed Publish/Subscribe Event Notification HTTP is limited to request/response semantics May be significant protocol overhead Arguably a limitation of HTTP 1.1, not REST

Examples of fitting publish/subscribe into HTTP: Simulating an Event Stream as a long HTTP GET Request Installing an HTTP Server on the Agent side

No obvious message correlation, hindering visibility

86

Events on the Web: Comet-Style AJAX Applications

Examples include: Google Mail’s GTalk

integration Meebo instant messaging Renkoo Jot Live

Simulate events through long HTTP GET requests

References

88

Further Reading

Andreessen, Mark. IIOP and the Distributed Objects Model http://web.archive.org/web/19961026035558/http://www3.netscape.com/comprod/columns/techvision/iiop.html

Box, Don. Lessons from the Component Wars: An XML Manifesto http://msdn2.microsoft.com/en-us/library/aa468562.aspx

Fielding, Roy. Architectural Styles and the Design of Network-Based Software Architectures http://www.ics.uci.edu/~fielding/pubs/dissertation/top.htm

Fielding, Roy. ApacheCon 2007 Presentation: The Rest of REST http://roy.gbiv.com/talks/200709_fielding_rest.pdf

The Object Management Group, The Common Object Request Broker: Architecture and Specification http://www.omg.org/docs/formal/99-10-07.pdf

The OpenID Foundation http://openid.net

Orfali, Harkey, and Edwards. BYTE Magazine (October 1997), CORBA, Java, and the Object Web http://www.byte.com/art/9710/sec6/art3.htm

Maier & Rechtin. The Art of Systems Architecting. CRC Press, ISBN 0849304407

Nieslen & LaLiberte. Editing the Web: Detecting the Lost Update Problem Using Unreserved Checkout http://www.w3.org/1999/04/Editing/

Nottingham, Mark. Post Once Exactly (POE) Internet Draft http://www.mnot.net/drafts/draft-nottingham-http-poe-00.txt

Richardson & Ruby. RESTful Web Services. O’Reilly Media, ISBN 0596529260

Russell, Alex. Comet: Low Latency Data for the Browser http://alex.dojotoolkit.org/?p=545

World Wide Web Consortium, Architecture of the World Wide Web, Volume One http://www.w3.org/TR/webarch/