1 The Endeavour Expedition: Computing and Communications at the eXtremes Professor Randy H. Katz IBM...
-
date post
19-Dec-2015 -
Category
Documents
-
view
214 -
download
0
Transcript of 1 The Endeavour Expedition: Computing and Communications at the eXtremes Professor Randy H. Katz IBM...
1
The Endeavour Expedition: Computing and Communications at the eXtremes
Professor Randy H. KatzIBM Research Visit
15 June 2000405 Soda Hall
2
The eXtremes
Scalable, AvailableInternet ServicesInformation
Appliances
Client
Server
Clusters
Massive Cluster
Gigabit Ethernet
MEMS
The Very LargeThe Very Small
New System ArchitecturesNew Enabled Applications Diverse, Connected, Physical
3
Presentation Outline
• Expedition Motivation– The Needed Revolution in Computing and Communications
• Packing for the Expedition– Project Organization
• Base Exploration plus Extended Expeditions• Summary and Conclusions
4
Background
• PITAC Report: “Information Technology Research--Investing in Our Future”
– Create a strategic initiative in long-term information technology R&D
– Priorities: Software, Scalable Information Infrastructure, High-End Computing, Socioeconomic Impacts
• DARPA (and Industry) as Patron– Chart potential “revolutions” in information technology,
with promise to achieve dramatic improvements in computing and applications in 21st Century
– Technology discontinuities drive new computing paradigms, applications, system architectures
» E.g., Project MAC, DARPAnet, Xerox Alto– What will drive the next discontinuity?
5
The Endeavour Expedition:Motivation and Goals
• Exploiting IT to enhance understanding– Make it dramatically more convenient for people to
interact with information, devices, and other people– Supported by a “planetary-scale” Information Utility
» Stress tested by challenging applications in decision making and learning
» New methodologies for design, construction, and administration of systems of unprecedented scale and complexity
– Figure of merit: how effectively we amplify and leverage human intellect
• Technical Approach: Pervasive Information Utility, based on “fluid systems,” enabling new approaches for problem solving & learning
6
Why “Endeavour”?
• To strive or reach; a serious determined effort (Webster’s 7th New Collegiate Dictionary); British spelling
• Captain Cook’s ship from his first voyage of exploration of the great unknown of his day: the southern Pacific Ocean (1768-1771)
– Brought more land and wealth to the British Empire than any military campaign
– Cook’s lasting contribution: comprehensive knowledge of the people, customs, and ideas that lay across the sea
– “He left nothing to his successors other than to marvel at the completeness of his work.”
• Software and resources as a fluid: pools, floods, rivers, eddies, containers, ...
7
Signing On to the Expedition
• “Difficulties are just things to overcome.”
• "Men [and Women] wanted for Hazardous Journey. Small wages, bitter cold, long months of complete darkness, constant danger, safe return doubtful. Honour and recognition in case of success."
– Sir Ernest Shackleton, Legendary Antarctic explorer who lost not a single person on two highly perilous expeditions to reach the South Pole (which he never reached!)
• Business UNusual: Research seminars, experimental courses, new synthesis of existing courses
8
Expedition Assumptions
• Human time and attention, not processing or storage, are the limiting factors
• Givens:– Vast diversity of computing devices (PDAs, cameras, displays,
sensors, actuators, mobile robots, vehicles); No “average” device
– Unlimited storage: everything that can be captured, digitized, and stored, will be
– Every computing device is connected in proportion to its capacity
– Devices are predominately compatible rather than incompatible (plug-and-play enabled by on-the-fly translation/adaptation)
• Tremendous innovations in technology; yet a comprehensive system architecture is lacking
9
Expedition Challenges
• Managing Attention is the Killer App– Not corporate processing but management, analysis,
aggregation, dissemination, filtering for the individual
• Create Knowledge, not Data– Not management/retrieval of explicitly entered
information, but automated extraction and organization of daily activities
• Information Technology as a Utility– Continuous service delivery, on a planetary-scale, on
top of a highly dynamic information base
• Beyond the Desktop– Community computing: infer relationships among
information, delegate control, establish authority
10
The Coming Revolution
Evolution
Information Appliances:Scaled down desktops,e.g., CarPC, PdaPC, etc.
Evolved Desktops
Servers:Scaled-up Desktops,
Millennium
Revolution
Information Appliances:Many computers per person,
MEMs, CCDs, LCDs, connectivity
Servers: Integrated withcomms infrastructure;Lots of computing in
small footprint
Display
Keyboard Disk
Mem
Proc
PC Evolution
Display Display
Camera
Sm
art
Senso
rs
Camera
Smart Spaces
ComputingRevolution
WAN
Server, Mem, Disk
InformationUtility
BANG!
Display
Mem
Disk
Proc
11
Expedition Approach• Information Devices
– Beyond desktop computers to MEMS-sensors/actuators with capture/display to yield enhanced activity spaces
• InformationUtility
• InformationApplications
– High Speed/Collaborative Decision Making and Learning
– Augmented “Smart” Spaces: Rooms and Vehicles
• Design Methodology– User-centric Design with
HW/SW Co-design;– Formal methods for safe and trustworthy
decomposable and reusable components
“Fluid”, Network-Centric System Software
– Partitioning and management of state between soft and persistent state
– Data processing placement and movement
– Component discovery and negotiation
– Flexible capture, self-organization, and re-use of information
12
The Endeavournauts:Interdisciplinary,
Technology-Centered Expedition Team
• Alex Aiken, PL• Eric Brewer, OS**• John Canny, AI• David Culler, OS/Arch• Michael Franklin, DB*• Joseph Hellerstein, DB• Michael Jordan, Learning*• Anthony Joseph, OS• Randy Katz, Nets• John Kubiatowicz, Arch• James Landay, UI
• Jitendra Malik, Vision• George Necula, PL*• Christos Papadimitriou, Theory• David Patterson, Arch• Kris Pister, Mems**• Larry Rowe, MM• Alberto Sangiovanni-
Vincentelli, CAD• Doug Tygar, Security• Robert Wilensky, DL/AI
*Speaking in New Faculty Perspectives Session** Speaking in Keynote Session
13
Organization: The
Expedition Cube
Information Devices
Information Utility
ApplicationsDesIgn
Methodology
MEMS Sensors/Actuators, Smart Dust, Radio Tags, Cameras, Displays, Communicators, PDAs
Fluid Software, Cooperating Components,Diverse Device Support, Sensor-CentricData Mgmt, Always Available, TacitInformation Exploitation (event modeling)
Rapid Decision Making, Learning,Smart Spaces: Collaboration Rooms,Classrooms, Vehicles
Base ProgramOption 1: Sys Arch for Diverse DevicesOption 2: Oceanic Data Utility
Option 4: Negotiation Arch for CooperationOption 5: Tacit Knowledge InfrastructureOption 6: Classroom TestbedOption 7: Scalable Safe Component-Based Design
Option 3: Capture and Re-Use
14
Base Expedition: Leader Katz
• Broad but necessarily shallow investigation into all technologies/applications of interest
– Primary focus on Information Utility» No new HW design: commercially available information
devices» Only small-scale testbed in Soda Hall
– Fundamental enabling technologies for Fluid Software» Partitioning and management of state between soft and
persistent state» Data and processing placement and movement» Component discovery and negotiation» Flexible capture, self-organization, info re-use
– Limited Applications– Methodology: Formal Methods & User-Centered Design
15
System Architecture for Vastly Diverse Devices
Leader Culler
• Design Issues for “Small Device OS”– Current: managing address spaces,thread scheduling,
IP stack, windowing system, device drivers, file system, APIs, power management
– How can OSs for tiny devices be made radically simpler, manageable, and automatically composable?
• Devices of Interest: Dust Motes
16
Servers
ClientsClients
ClientsClientsClients
ClientsServers
Servers
Infrastructure Services
Open
“The Large”: Service-Centric Platform Arch
• Enable distributed creation/deployment of scalable, available services
– Service registry, aggregate execution env., transparency – Persistent distributed data structures– Massive fluid storage (“Oceanic” Storage)– Adaptive high-bandwidth flows (rivers)
• Build infrastructure via composition of services
17
“The Small”: Radically Simple OS for Management &
Composition• Basic Assumptions:
– Communication is fundamental– Direct “user interface” is the exception not the norm– Critical resource is scheduling data movements, not arbitrary
threads of computation
• Tiny OS: Little more than an FSM– Commands: event stream merged with sensor/actuator events– General thread compiled to sequence of bounded atomic
xacts– Constant self-checking and telemetry– Rely on the infrastructure for complex processing
• Correctness-by-construction techniques for cooperating FSMs (tie in to HW/SW co-design)
18
Pac Bell
IBM
AT&T
CanadianOceanStore
IBM
Sprint
Implementation & Deployment of Oceanic
Data Info UtilityLeader Kubiatowicz
• Ubiquitous devices require ubiquitous storage
– Consumers of data move, change access devices, work in many different physical places, etc.
• Needed properties:– Strong Security– Coherence– Automatic replica management
and optimization– Simple and automatic recovery
from disasters– Utility model
• Confederations of (Mutually Suspicious) Utilities
19
OceanStore Architecture/Technology
• Name and Data Location– Issue: Find nearby data without global communication– Approach: Data location is aform of gradient-search of local pools of data (use
of attenuated Bloom-filters)
• High Availability and Disaster Recovery– Issue: Eliminate backup as independent/fallible technology– Approach: Erasure-codes/mobile replicas provide stable storage for archival
copies and snapshots of live data
• Introspective Monitoring and Optimization– Issue: Optimize performance on a global scale– Approach: Monitoring and analysis of access/usage relationships
• Rapid Update in Untrusted Infrastructure– Issue: Updates should not reveal info to untrusted servers– Approach: Incremental cryptographic techniques/oblivious function
techniques to perform update
20
Sensor-Centric Data Management for Capture/Reuse
Leader Hellerstein• Managing Data Floods
– Never ends: interactive direction– Big: data reduction/aggregation– Unpredictable: scale of devices
and nets not behave nicely
• Builds on CONTROL and River/Eddy System
– Early answers, interactivity, online aggregation
– Information processing via massively parallel, adaptive dataflows
– Extended to wide-area: operator placement, reordering
• Telegraph Data Manager– Distributed Storage Manager
based on event flow and state machines
– Continuously adaptive dataflow with applications to sensor data and streaming media
21
Negotiation Architecture for Cooperating Components
Leader Wilensky
• Cooperating Components– Self-administration through auto-discovery and
configuration among confederated components– Less brittle/more adaptive systems
• Negotiation Architecture– Components announce their needs and services– Service discovery and rendezvous mechanisms to
initiate confederations– Negotiated/contractural APIs: contract designing
agents– Compliance monitoring/renegotiation/non-
compliance recovery– Graceful degradation in response to
environmental changes
22
Tacit Knowledge Infrastructureand Collaborative Applications
Leaders Canny/Joseph/Landay
• Exploit information about the flow of information to improve collaborative work
– Capture, organize, and place tacit information for most effective use
– Learning techniques: infer communications flow, indirect relationships, availability/participation to enhance awareness and support opportunistic decision making
• New applications– 3D “activity spaces” for representing decision-making
activities, people, & information sources– Visual cues to denote strength of ties
between agents, awareness levels, activity tracking, & attention span
– Electronic Problem-based Learning in Enhanced Physical and Virtual Learning Spaces
Display Display
Camera
Sm
art
Senso
rs
Camera
Smart Spaces
23
User Interfaces and Design Tools
James Landay and John Canny
• Future devices won’t have PC-style UI – Extreme range of devices
» Small or embedded in environment» Often w/ “alternative” I/O & w/o screens » Special purpose applications, e.g., Info Appliances
– Lots of devices per user, all working in concert
• Key Technologies– Tacit information analysis algorithms– Design tools that integrate
» “Sketching” & other low-fidelity techniques» Immediate context & tacit information» Interface models
24
Safe Component Design Leader Sangiovanni
• Correctness by Construction– Safe partitionings of communicating subcomponents
placed in wide-area– Builds on on-going work in embedded systems design
• Compositions of Third Party Components– Safety enforcement technologies– Design and development methodologies – Builds on Necula’s Proof Carrying Code
• Trust and Assurance– Integrated use of secure tokens for rights management,
economic protocols/auctions, support for mobile code, ...– Secure protocol design & deployment based on super-
fast model-checking/automatic generation from requirements
25
Experimental Testbeds
NetworkInfrastructure
GSMBTS
Millennium Cluster
Millennium Cluster
WLAN /Bluetooth
Pager
IBMWorkPad
CF788
MC-16
MotorolaPagewriter 2000
Velo
TCI @HomeAdaptive Broadband LMDS
H.323GW
Nino
Smart ClassroomsAudio/Video Capture Rooms
Pervasive Computing LabCoLab
Soda Hall
CalRen/Internet2/NGI
Smart
DustLCD Displays
WearableDisplays
26
Putting It Together
1. eXtreme Devices
2. Data Utility
3. Capture/Reuse
4. Negotiation
5. Tacit Knowledge
6. Classroom
7. Design Methods
8. Scale-up
Devices
Utility
Applications
Fluid Software
Info Extract/Re-use
Decision MakingGroup Learning
Component Discovery& Negotiation
Self-Organization
27
Summary and Conclusions
• 21st Century Computing– Making people’s exploitation of information more effective– Encompassing eXtreme diversity, distribution, and scale– Computing you can depend on
• Key Support Technologies– “Fluid software” computational paradigms– System and UI support for eXtreme devices– Pervasive, planetary-scale system utility functionality– Active, adaptive, safe and trusted components – New “power tool” applications that leverage community activity
• Broad multidisciplinary team spanning the needed applications, evaluation, and system technology skills
– Culture of large-scale, industry-relevant high impact research projects