Secure Location-Secure Location-Independent Autonomic Independent Autonomic Storage ArchitecturesStorage Architectures

GR/S44501/01GR/S44501/01February 2004 - January 2007February 2004 - January 2007

Graham Kirby, Alan Dearle, Ron Morrison & Stuart Graham Kirby, Alan Dearle, Ron Morrison & Stuart NorcrossNorcross

School of Computer Science, University of St AndrewsSchool of Computer Science, University of St Andrews{graham, al, ron, stuart}@dcs.st-and.ac.uk{graham, al, ron, stuart}@dcs.st-and.ac.uk

EPSRC e-Science 26/3/04

2

Project AimsProject Aims Desirable features of a data storage systemDesirable features of a data storage system

unbounded capacityunbounded capacity zero latency & costzero latency & cost total reliabilitytotal reliability location independencelocation independence simple interfacesimple interface complete securitycomplete security complete historical archivecomplete historical archive

Aim: a storage architecture approximating above, focusing Aim: a storage architecture approximating above, focusing on:on: simple interface for end user (file system)simple interface for end user (file system) abstracting over:abstracting over:

user locationuser location physical devicesphysical devices

provision of significant benefits with acceptable costprovision of significant benefits with acceptable cost

EPSRC e-Science 26/3/04

3

Potential BenefitsPotential Benefits

Simplify user experienceSimplify user experience ‘‘home directory’ ubiquitously available, irrespective of:home directory’ ubiquitously available, irrespective of:

machines and disksmachines and disks physical locationphysical location firewallsfirewalls

data highly durabledata highly durable no need for backupno need for backup

simple data sharingsimple data sharing uniform global name spaceuniform global name space

Historical viewsHistorical views data never over-writtendata never over-written

EPSRC e-Science 26/3/04

4

Potential Hurdles to User AdoptionPotential Hurdles to User Adoption Speed and convenience must be close enough to Speed and convenience must be close enough to

that of a local diskthat of a local disk Users must be able to trust systemUsers must be able to trust system

not to allow inappropriate access to data by other usersnot to allow inappropriate access to data by other users to be sufficiently reliable for serious evaluationto be sufficiently reliable for serious evaluation

Need viable exit strategyNeed viable exit strategy may require that system can reproduce effects of user’s may require that system can reproduce effects of user’s

existing backup regimeexisting backup regime e.g. by maintaining a local copy of all datae.g. by maintaining a local copy of all data

Financial costFinancial cost Critical mass of nodes and users requiredCritical mass of nodes and users required

envisaged architecture relies on autonomic management envisaged architecture relies on autonomic management of large numbers of nodesof large numbers of nodes

Storage overhead must be low enoughStorage overhead must be low enough incurred through replication of dataincurred through replication of data

EPSRC e-Science 26/3/04

5

User ControlUser Control End users should deal only with very high-level End users should deal only with very high-level

configurationconfiguration set broad goals regarding trade-offs (or ignore set broad goals regarding trade-offs (or ignore

completely)completely) task of autonomic management system to try to achieve task of autonomic management system to try to achieve

these goalsthese goals

Examples of trade-offsExamples of trade-offs speed of reads and writesspeed of reads and writes durabilitydurability

related to number and placement of replicasrelated to number and placement of replicas both absolute & time to convergeboth absolute & time to converge

consistencyconsistency how long before updates to shared data are visible to others?how long before updates to shared data are visible to others?

resource consumptionresource consumption storage, bandwidth, computationstorage, bandwidth, computation

EPSRC e-Science 26/3/04

6

Control ExampleControl Example

EPSRC e-Science 26/3/04

7

Control and Feedback ExampleControl and Feedback Example

EPSRC e-Science 26/3/04

8

Implementation ApproachImplementation Approach

File system interfaceFile system interface SMB or NFSSMB or NFS

Replication of files or fragmentsReplication of files or fragments erasure-resilient encodingerasure-resilient encoding

Placement of dataPlacement of data controlled explicitlycontrolled explicitly

Routing to dataRouting to data abstracted by peer-to-peer overlay e.g. Tapestryabstracted by peer-to-peer overlay e.g. Tapestry

Probes & gauges to monitor state of systemProbes & gauges to monitor state of system publish/subscribe infrastructure e.g. Siena publish/subscribe infrastructure e.g. Siena

Autonomic management elementsAutonomic management elements attempt to map user goals and probe events into attempt to map user goals and probe events into

suitable low-level actionssuitable low-level actions

EPSRC e-Science 26/3/04

9

ChallengesChallenges

Core distributed storage infrastructureCore distributed storage infrastructure appropriate replication mechanismsappropriate replication mechanisms

Autonomic managementAutonomic management low-level policieslow-level policies probe & gauge infrastructureprobe & gauge infrastructure high-level views for usershigh-level views for users

synthesising views from low-level eventssynthesising views from low-level events heuristics for adapting low-level policies to achieve high-heuristics for adapting low-level policies to achieve high-

level goalslevel goals

EvaluationEvaluation simulation, local cluster, PlanetLabsimulation, local cluster, PlanetLab end-user adoptionend-user adoption

EPSRC e-Science 26/3/04

10

ConclusionsConclusions

Aim to design, implement and evaluate Aim to design, implement and evaluate distributed storage system targeted at benefits to distributed storage system targeted at benefits to end-userend-user very simple interfacevery simple interface ubiquitously availableubiquitously available highly durablehighly durable append-only: historical viewsappend-only: historical views

Project detailsProject details http://www-systems.dcs.st-and.ac.uk/asa/http://www-systems.dcs.st-and.ac.uk/asa/