1

WebDAV and DeltaV: Collaborative Authoring, Versioning,

and Configuration Management for the Web

Jim WhiteheadUniversity of California, Santa Cruzejw@cs.ucsc.edu

2

WebDAV and DeltaV

WebDAV and DeltaV are: Application layer network protocols Extensions to the Hypertext Transfer Protocol (HTTP

1.1) Developed by the Internet Engineering Task Force

(IETF)

WebDAV Remote collaborative authoring of Web resources Overwrite prevention, metadata management,

namespace operations

DeltaV Versioning and Configuration Management Extends base WebDAV capabilities Checkout/checkin, version history, logical change

tracking, workspaces, configuration management

3

Major WebDAV Clients Application Software:

Microsoft: Office 2000/XP (Word, Excel, PowerPoint, Publisher)

Adobe: Photoshop 6, Illustrator 10, Acrobat 5, In Design 2 Web Site Authoring

Adobe: Go Live 5 Macromedia: Dreamweaver 4

Remote File Access: Apple: Mac OS X webdavfs

OS X also ships with Apache and mod_dav (have to configure mod_dav to make it work)

Microsoft: Windows Web Folders Wind River Software: WebDrive Goliath (Mac, open source) WebDAV Explorer (UC Irvine, Feise/Kanomata, open source)

XML editors Excosoft: Documentor Altova: XML Spy SoftQuad: XMetal

4

Major WebDAV ServersMicrosoft: IIS 5/6, Exchange 2000, SharepointApache: mod_dav (over 95,000 sites)Oracle: Internet File SystemAdobe: InScopeXythos: Web File ServerNovell: Netware 5.1, Net PublisherW3C: JigsawEndeavors: Magi-DAVIBM: DAV4J (DeveloperWorks)FileNet: Panagon ECMIntraspect: 4iMerant: PVCS Dimensions, Content ManagerHyperwave: Information Server 5.54D: WebSTAR V

5

Collaborative Document Authoring

Three collaborators, in different cities, use Word 2000 to collaborate on a report they are producing together.

WebDAV server

A

Johnathan

Word 2000

BOSTON LOS ANGELES

shared document

Katrina Word 2000

TOKYO

Kenji

Word 2000

HTTP/DAV HTTP/DAV

HTTP/DAV

6

Collaborative Web Site Authoring

Two homes each develop their family Web site using WebDAV to interact with their ISP.

WebDAV server

Joe Homeuser Go Live 5

/joe

/html /images

HTTP/DAV

INTERNET SERVICE PROVIDER HOME 1

/jane

/html /images

Jane McPC DreamWeaver

4

HTTP/DAV

Jane’s web site

HOME 2

Joe’s Daughter

Go Live 5

Web site for Joe and his daughter

7

Remote Authoring, Part of Staged Production

WebDAV server

Frank

Excel 2000

CHICAGO SEOUL

Yung Kim Go Live

5

FRANKFURT

Hartmut

Photo shop 6

HTTP/DAV HTTP/DAV

HTTP/DAV

/contrib

/seoul /frankfrt project Web site

Cindy

Acrobat 5

/chicago

contributions server

HTTP server

development server

web site contents

Web site development team incorporates contributions submitted to the contributions server.

HTTP server

web site contents

General Internet Readers

firewall

production server

8

Visions for WebDAV

Participants in WebDAV have many views on what it is: A protocol for collaborative authoring of all document types

XML, HTML, word processing, spreadsheets, A Web-based network file system, with nice high-latency

behavior Better performance than NFS and Samba over the Internet

A data integration technology for accessing a wide range of repositories

Document mgmt. systems, configuration mgmt. systems, filesystems, etc.

Remote software engineering infrastructure A replacement protocol that can handle email, calendaring,

directory lookup and more Could replace: POP, IMAP, CAP, LDAP…

All views are correct!

9

Filesystem View

Exemplars: Web Folders, WebDrive, WebIFS, TeamDrive, Mac OS X

10

Document Authoring Exemplars: Office 2000/XP: Word, Excel, PowerPoint,

as well as Photoshop, Documentor, and XML Spy

Office: uses filesystem metaphor for WebDAV location

11

Photoshop Workflow metaphor for WebDAV location

12

Web Site Authoring Exemplars: Go Live 5/6, Dreamweaver Site metaphor for WebDAV location

13

Remote Collaborative Annotation Acrobat 5 views a WebDAV location as a storage location

for document annotations Annotations are stored in resources separate from the PDF

document One collection per document One annotation resource per user (in collection)

14

Facets of WEBDAV

There are many ways to view the DAV work: Collaboration infrastructure Metadata repository infrastructure Namespace management infrastructure Access control infrastructure Searching infrastructure – DASL

15

Collaboration Infrastructure Whole resource locking supports:

remote collaborative authoring of any media type Web pages, Word processing, Presentations, XML, …

Lock characteristics: Long-duration locks Not associated with network connection

Client receives a lock token identifying the lock Can disconnect from network after receiving lock token

Locks automatically expire after a client specified timeout period

Lock single resources, or hierarchies of resources Infrastructure for asynchronous, widely

distributed, hypertext-aware, collaborative editing tools.

16

Metadata Recording Infrastructure

Metadata support Properties are (name, value) pairs that

can be created, modified, deleted, and read on Web resources.

Consistency of properties can be maintained by the server or the client

Property values are well-formed Extensible Markup Language (XML) Can store RDF as well

Property name is a URI or URL Extensible, global property namespace

Infrastructure for recording information about Web data A general purpose metadata repository

18

Access Control Infrastructure

Access Control: The ability to remotely control who can read and write a

resource Key challenges:

Expose the access control capabilities of the repository… …while ensuring the client-side user interface can be simple

(I.e., avoid lots of feature discovery) Provide some notion of principal, without duplicating LDAP Handle large numbers of principals

Access control lists Each entry grants/denies a privilege to a principal (or group)

Infrastructure for remotely creating collaboration groups

19

Searching Infrastructure

Searching a WebDAV repository - DASL: Search for resources with a given property, or

a given property value Search for a substring inside a resource body Search scope can be one resource, a

collection of resources, a hierarchy of resources, or a whole server

Search syntax is extensible Search specification specified as body

submitted with SEARCH command Could accommodate XML Query syntax

Infrastructure for remote searching

20

WebDAV Methods

and Data Model

21

WebDAV Methods

Overwrite Prevention: LOCK – prevents non-lock holders from writing to the

resource UNLOCK – removes a lock

Metadata Management: PROPFIND – read properties from a resource

Allprop – all property names and values Propname – only return property names Prop – just return specified properties

PROPPATCH – write properties on a resource

Namespace Management COPY – duplicate a resource MOVE – move a resource (preserving identity) MKCOL – create a new collection

22

Scope of WebDAV Methods

Web Resource

Body(primary

state)

Properties(name, value)

pairsLOCKUNLOCKCOPYMOVE†

DELETE†

MKCOL†

(PUT†)

PROPFIND

PROPPATCH†

GET

PUT†

† - affected by

LOCK

23

DeltaV: Versioning and Configuration Management

24

Remote Collaborative Document Authoring Scenarios

A task force of people from geographically dispersed business units need to develop a report together. Team needs to solicit feedback, so they send out

copies of the report. As a result, the team needs a permanent copy of the

exact report version they are having other people review.

DAV/DeltaV server

A

Johnathan

Word 2000

BOSTON LONDON

shared document

Rebecca

Word 2000

TOKYO

Kenji

Word 2000

DAV/DeltaV DAV/DeltaV

DAV/DeltaV

A A

v1 v2 v3

auto-versioning enabled

25

What is DeltaV?

DeltaV is: An application layer network protocol

Extends the WebDAV protocol WebDAV itself extends HTTP 1.1

Thus DeltaV is also an extension to HTTP 1.1 HTTP is the core network protocol of the Web

An interoperability standard (RFC 3253) Allows document authoring and software development

applications to interoperate with versioning and configuration management systems using a common interface

A data model DeltaV embodies a data model that is capable of modeling

multiple versioning and configuration management repositories

The data model is independent of network protocol A data integration infrastructure

A common place for cross-application data sharing

26

Version History

Foo.htm

1

2

7

65

43

initial

Beta1

Test1

Beta2

URL path of VersionedControlledResource

Version Name

Label

Successor

Line ofDescent

RevisionHistory

Predecessor

Branch

Merge

27

Representation of a Version History

v1

v2

v3

Abstract version history of foo.html

Representation of foo.html in DeltaV

v1 v2 v3

version resources

body of version v3

/his/73/ver/1 /his/73/ver/2 /his/73/ver/3

version-controlled resource

/foo.html

DAV:checked-out /his/73/ver/3

version history resource

DAV:version-history /his/73/vhr

/his/73/vhr

DAV:version-set /his/73/ver/1 /his/73/ver/2 /his/73/ver/3

DAV:root-version /his/73/ver/1

28

Autoversioning There are many WebDAV-aware applications

They know nothing about versioning! Would like to provide automatic versioning support

Two styles of autoversioning:1. Every modification creates a new version

PUT/PROPPATCH CHECKOUT PUT/PROPPATCH CHECKIN

Works best for authoring clients that replicate resources to a local disk, and only write to the server at the end of an editing session.

2. Every LOCK/UNLOCK pair creates a new version LOCK LOCK CHECKOUT UNLOCK CHECKIN UNLOCK Works well for authoring clients that work directly on

the WebDAV server and take out locks, like Office 2000. Style is controlled by DAV:autocheckout and

DAV:autocheckin properties on version controlled resource

29

Workspaces

A workspace is a location… … where a person can work in isolation … from ongoing changes made by other collaborators to the same set of resources.

Workspace use scenarios: Two people each modify a common source code file,

and want to test their work before integrating changes The directory structure of a large project is being

changed Don’t want to require everyone to stop working while this

takes place Keep namespace changes invisible to team until complete

A developer is trying a change that might not pan out New files are in their workspace, but not under version

control until the final viability of the change is known

30

Server Workspace Example Version history of main.c:

1 2 3 4

/repo/o522/v1

/repo/o522/v2

/repo/o522/v3

/repo/o522/v4

version history resource

/repo/o522/v1

/repo/o522/v2

/repo/o522/v3

/repo/o522/v4

version resources

/his/o522

/projectX/main.c

/users/chris/ projectX/main.c

/users/geoff/ projectX/main.c

version controlled resources

5

/repo/o522/v5

31

Server Workspace Example

main.c

2

makefile

3

1

4

2

3

1

defs.h

2

1

/users/chris/projectX/main.c, 4makefile, 3defs.h, 3

/users/geoff/projectX/main.c, 5makefile, 3defs.h, 2

3

5

33

WebDAV Projects at UC Santa Cruz

34

Recontextualizing WebDAV

WebDAV and DeltaV gathered decontextualized requirements for collaboration and version control Abstracted away characteristics of specific application

domains Used Web authoring & software development as driving

problems

Led to a powerful, inexpensive, generic content management capability

Now, goal is to apply the general-purpose capabilities of WebDAV/DeltaV to solve specific problems: Electronic records management Digital libraries … as well as ensuring they satisfy original goals: Document management Software development

35

Advanced Document Collaboration Infrastructure

Goal: Create advanced open-source WebDAV server with following characteristics: Highly scalable

store and search millions of documents Data integration

knit together multiple document and metadata repositories

Access Postdoc, legacy stores via DAV/DeltaV Access Control

support the emerging ACL protocol standard Versioning

document versioning and auto-versioning support

36

Advanced Document Collaboration Infrastructure (2)

Peter

Word

NASA Ames NASA Goddard

Jason PowerPoint

JPL

Julia

Windows Web Fldr

HTTP/DAV HTTP/DAV

HTTP/DAV

Postdoc repository

SQL database

legacy metadata

store

Apache

mod_dav mod_dav_repos

wrapper

wrapper

wrapper

37

Advanced Document Collaboration Infrastructure (3)

Research challenges: Abstract repository interface

Develop API between mod_dav_repos and repository wrappers

Scalability Ensuring the architecture supports large numbers of

documents and storage Supports efficient searching

Cross-repository searching Support for efficient DASL searches Accommodating server-defined and client-defined

properties Allowing searches that span the contents of multiple

repositories Access Control Implementation

Create an open reference implementation of the access control protocol

Develop an access control client that can adapt to multiple access privilege hierarchies

38

Advanced Document Collaboration Infrastructure (4)

Work performed so far: Created mod_dav_dasl prototype

Single database, MySQL Storage and searching of arbitrary dead properties Implemented DASL Some performance characterization Client support for DASL in cadaver client Work is underway to release the code

Work performed by Elias Sinderson (also working at ARC), Sung Kim, Kai Pan

39

Electronic Records Management US Government agencies are required to

support standard DoD 5015.2 Describes metadata items useful for archiving

electronic records 5015.2 is not an interoperability standard

Approach Develop an XML representation of 5015.2 metadata

items Store 5015.2 metadata in WebDAV properties Develop server support for

automatic archiving/disposal of records automatic setting of 5015.2 property values

Develop client support for entry/viewing/searching 5015.2 items

Work performed by Dorrit Gordon, Leila Naslavsky

40

Electronic Records Management (2)

Work performed so far: Preliminary XML metadata schema for 5015.2

metadata items Modular approach allows work to potentially be used for

ERM in other countries (e.g., VERS standard in Australia) Initial work on identifying items with controlled

vocabularies Containment data modeling of 5015.2 data model

Payoff Wide deployment of interoperable, low-cost

infrastructure for archiving electronic records Infrastructure for personal records management

In the future, how will you manage that 10TB hard drive on your PC?

41

Personal Digital Library

Researchers interact with a number of institutional digital libraries ACM and IEEE are significant for CS researchers

Limited existing support for users interacting with these institutional DLs Either read files online, or Store files in a filesystem

Institutional DLs have poor track record for reliability, QoS Users want to have local copies of relevant documents

Disk sizes are large, and growing It’s reasonable to keep a copy of every document ever

read How do you manage all these documents?

42

Personal Digital Library (2)

Most researchers know other researchers in their area If another researcher in your area finds an interesting

paper, you’ll want a copy This should happen automatically, just by being a

member of a research peer group Our approach:

Use a WebDAV server to store a personal, or small workgroup’s digital library

Augment documents with bibliographic metadata Special-purpose document browsing and searching

client Provides a document UI, not a file UI Emphasizes bibliographic metadata in browsing UI

Integrate P2P capability for document sharing User-selectable policies for IP enforcement

Current status: NSF proposal in-progress

43

Adaptive Configuration Management Client

DeltaV specification defines multiple packages (feature sets) of functionality Core versioning Basic server workspace Basic client workspace Advanced server workspace Advanced client workspace

A server may support one or more of these Research challenge:

Create a DeltaV client that can adapt to any server it encounters

Implies it can adapt to different workspace semantics, and different supported features

Existing CM clients are tailored to a single system Status: looking for a sponsor

44

Software Engineering at UC Santa Cruz

45

Software Engineering @ UCSC

Currently in the process of building up an SE research group Hiring for three SE faculty this year

In-process of developing a Masters degree program in Software Engineering Aimed at working professionals in Silicon Valley Goal is to eventually offer many courses at the UC

Silicon Valley Center (on the Ames campus) Coursework in Software Engineering plus a significant

multi-quarter (perhaps including summer) project

Also developing a Masters program in Web and Internet Engineering Courses in SE, databases, networking, storage

systems Also aimed at SV working professionals

48

WebDAV Resources

49

WebDAV WebDAV Resources

http://www.webdav.org/ A central collection of pages and links to all things WebDAV.

WebDAV Working Group http://www.ics.uci.edu/pub/ietf/webdav/

Contains links to active documents, and a complete list of WebDAV-supporting applications.

RFC 2518 – WebDAV Distributed Authoring Protocol http://www.ics.uci.edu/pub/ietf/webdav/protocol/rfc2518.pdf

This is the WebDAV Distributed Authoring Protocol specification

WebDAV: A network protocol for remote collaborative authoring on the Web Proc. of the Sixth European Conference on Computer-

Supported Cooperative Work, Sept. 12-16, 1999, Copenhagen, Denmark, pp. 291-310.

http://www.ics.uci.edu/~ejw/papers/dav-ecscw.pdf An academic paper giving an overview of the WebDAV

Distributed Authoring Protocol.

50

DeltaV Delta-V Working Group web page

http://www.webdav.org/deltav/ The home page for the IETF Delta-V Working Group, with

links off to the most recent specifications.

RFC 3253 – DeltaV Protocol http://www.webdav.org/deltav/protocol/rfc3253.html

Extensions to the WebDAV protocol (RFC 2518) for versioning and configuration management

The Future of Distributed Software Development on the Internet Web Techniques, Vol. 4, No. 10, October, 1999, pages

57-63 http://www.webtechniques.com/archives/1999/10/whitehead/

An introduction to WebDAV and DeltaV that describes the advantages of DeltaV over CVS for remote collaborative software development

51

DASL: DAV Searching & Locating

DAV Searching and Locating page http://www.webdav.org/dasl/

A web site containing links to the most recent WebDAV searching protocol specifications.

J. F. Reschke, S. Reddy, J. Davis, A. Babich, “WebDAV Search” http://greenbytes.de/tech/webdav/draft-reschke-

webdav-search-latest.html The most recently edited DASL protocol specification. Significantly increased momentum recently Likely to see specification approved this year.

52

Access Control Protocol

Access Control page http://www.webdav.org/acl/

A web site containing links to current access control protocol specifications.

WebDAV Access Control Protocol draft-ietf-webdav-acl-07, November, 2001. http://www.webdav.org/acl/protocol/draft-ietf-

webdav-acl-07.htm The most recent revision of the access control protocol

specification.

53

Advanced Collections

WebDAV Bindings draft-ietf-webdav-binding-protocol-02, December 17, 1999. http://www.ics.uci.edu/pub/ietf/webdav/collection/draft-ietf-

webdav-binding-protocol-02.txt The most recent revision of the WebDAV Bindings Protocol.

WebDAV Redirect Reference Resources draft-ietf-webdav-redirectref-protocol-02, December 17,

1999. http://www.ics.uci.edu/pub/ietf/webdav/collection/draft-ietf-

webdav-redirectref-protocol-02.txt The most recent revision of the Redirect Resources protocol.

WebDAV Ordered Collections Protocol draft-ietf-webdav-ordering-protocol-02, December 20, 1999. http://www.ics.uci.edu/pub/ietf/webdav/collection/draft-ietf-

webdav-ordering-protocol-02.txt The most recent revision of the Ordered Collections protocol.

54

Document Roadmap

Distributed AuthoringLocking, Properties, Copy/MoveRFC 2518 complete

Versioning & CMProtocol just sent to IESGRFC 3253 complete

Advanced CollectionsRequirements and protocol forbindings, redirectors, ordered coll.Finish:June 2003

Access ControlProtocol for remote access controlFinish: Spring 2002

WebDAV Working Group:

DASL:

SearchingRequirements and protocol forsearching a WebDAV repositoryFinish: late 2002

Delta-V Working Group:

55

56

Overwrite Prevention: Locking

57

Write Lock

A write lock prevents a principal without the lock from successfully modifying the state of the resource Specifically, it prevents execution of PUT,

POST, DELETE, MKCOL, PROPPATCH, MOVE, LOCK, UNLOCK

GET, PROPFIND are unaffected by write lock

58

Write Lock Scope The scope of a lock is an entire resource

It is not possible to specify a sub-resource lock. Why? The ability to create a sub-resource lock requires

knowledge about the content type being locked. WebDAV wanted to support all content types

equally, not give preferential support to a few. Due to the large number of content types, support

for sub-resource locks would have added a lot of extra semantics to the protocol. Some content types wouldn’t have been supported.

Content types change rapidly (two revisions of HTML and XML occurred during development of WebDAV protocol). Supporting one revision of a content type would make protocol brittle, quickly obsolete.

New content types would not be supported, yet new content types emerge constantly.

59

Write Lock Scope (3)

Implications of whole-resource locking: Pro:

Supports existing applications which operate on entire files, providing easy migration path to add WebDAV support

Handles all content types

Con: Reduced availability of resources during

collaboration (but, shared locks can help…)

60

Write Lock (cont’d)

Live properties may still change even though a lock is active

Dead properties may only be changed by the lock owner.

A null resource may be locked to reserve its name. This puts the resource into a special state, lock-null, since it now has lock related properties defined on it. In retrospect, a bad idea

Adds a lot of special case code Not a compelling feature for clients Feature may go away in next revision of RFC 2518

61

Lifecycle of a Lock

unlocked lockedLOCK

UNLOCK

LOCK(refresh)

lock timeout

administrator removal of lock

62

Exclusive Lock

Zero or one possible lock holders

lock holder

access permission holders

everyone on the Internet

63

Shared Lock

Zero, one, or many possible lock holders

lock holders

access permission holders

everyone on the Internet

64

Why Exclusive and Shared?

Exclusive locks are too rigid people often forget to release locks requires an administrator to release the lock

Shared locks allow people to use out-of-band communication to negotiate access to the resource if someone forgets to release a lock, it doesn’t

hold up the entire group

Collaborators work opportunistically

65

Lock Compatibility

Current lock state

SharedLock

Exclusive Lock

None

Shared Lock

Exclusive Lock

true

true

true

false

false false*

Legend: True = lock MAY be granted * = owner of lock MAYFalse = lock MUST NOT be granted have lock regranted

Lock request

66

LOCK Required Support

A WEBDAV server is not required to support locking

Why? Systems differ markedly in the type of locking support they provide, and may not be able to support locks at all (i.e., some replicated stores)

… However, most WebDAV servers do support locking

67

LOCK Method

LOCK creates the lock specified by the <lockinfo> XML element (in the request body) on the Request-URI.

Lock owner information is submitted with a lock request

A timeout value may also be requested

LOCK returns a lock token which uniquely identifies the lock to the server

68

Lock Owner Information

The owner XML element (inside <lockinfo>) provides a means to associate lock holder contact information with a lock. If you want the lock holder to release the lock,

perhaps you can contact them and ask them to relinquish it

Authentication information often does not contain contact information (e.g., a key)

69

LOCK, single resource

LOCK /webdav.html HTTP/1.1Host: sandbox.xerox.comTimeout: Second-500, InfiniteContent-Type: text/xmlContent-Length: 151

<?xml version="1.0" ?><d:lockinfo xmlns:d="DAV:"> <d:locktype><d:write/></d:locktype> <d:lockscope><d:exclusive/></d:lockscope> <d:owner>

<d:href>mailto:ejw@soe.ucsc.edu</d:href> </d:owner></d:lockinfo>

70

LOCK, single resource (2)

HTTP/1.1 200 OKDate: Tue, 09 Feb 1999 02:25:21 GMTServer: PyDAV 1.1 filestore 1.1Content-Type: text/xmlContent-Length: 435

<?xml version="1.0"?><A:prop xmlns:A="DAV:"> <A:lockdiscovery> <A:activelock> <A:lockscope> <A:exclusive/> </A:lockscope>

71

LOCK, single resource (3)

<A:locktype> <A:write/> </A:locktype> <A:depth>Infinity</A:depth>

<A:owner><A:href>mailto:ejw@soe.ucsc.edu</A:href> </A:owner> <A:timeout>Second-500</A:timeout> <A:locktoken> <A:href>opaquelocktoken:918527121.406</A:href> </A:locktoken> </A:activelock> </A:lockdiscovery></A:prop>

72

Hierarchy Locks

Using the Depth header set to Infinity, can lock a collection hierarchy A single lock token is returned, identifying the lock on

all resources. An UNLOCK on this token removes the lock from all

associated resources. All or nothing semantics

All resources in hierarchy are locked, or none are Hierarchy locks act to ensure:

All resources in the hierarchy are members of the lock

Resources removed from the hierarchy are removed from the lock

But… If a locked hierarchy is copied/moved, the destination

hierarchy is not locked.

73

UNLOCK

UNLOCK removes the lock identified by a lock token from the Request-URI, and all other resources included in the lock If a lock affects an entire collection, UNLOCK

removes the lock from all resources in the collection.