Post on 05-Jan-2016
Distributed Systems
Introduction
2
CS 403 Distributed Systems
D.S. Theory
Peer to peer systemsCloud Computing
Sensor Networks
The Hype! Forrester in 2010 – Cloud computing will go from
$40.7 billion in 2010 to $241 billion in 2020. Gartner in 2009 - Cloud computing revenue will
soar faster than expected and will exceed $150 billion by 2013. It will represent 19% of IT spending by 2015.
IDC in 2009: “Spending on IT cloud services will triple in the next 5 years, reaching $42 billion.”
Companies and even Federal/state governments using cloud computing now: fbo.gov
Many Cloud Providers• AWS: Amazon Web Services
– EC2: Elastic Compute Cloud– S3: Simple Storage Service– EBS: Elastic Block Storage
• Microsoft Azure• Google Cloud• Google Compute Engine• Rightscale, Salesforce, EMC, Gigaspaces,
10gen, Datastax, Oracle, VMWare, Yahoo, Cloudera
• And many many more!
Two Categories of Clouds Can be either a (i) public cloud, or (ii) private
cloud Private clouds are accessible only to company
employees Public clouds provide service to any paying
customer: Amazon S3 (Simple Storage Service): store arbitrary datasets, pay
per GB-month stored Amazon EC2 (Elastic Compute Cloud): upload and run arbitrary
OS images, pay per CPU hour used Google AppEngine/Compute Engine: develop applications within
their appengine framework, upload data that will be imported into their format, and run
Reference Texts
Andrew S. Tanenbaum and Marten van Steen: “Distributed Systems: Principles and
Paradigms” Prentice-Hall
George Coulouris, Jean Dollimore and Tim Kindberg: “Distributed Systems: Concepts and
Design” Addison-Wesley
7
Organization of the course
1 Characterization of DS 2 System Models 3 Networking and Internetworking 4 Interprocess Communication
Foundations
10 Time and Global States11 Coordination and Agreement
Distributed algorithms
6 Operating System Support 8 Distributed File Systems15 Distributed Multimedia Systems16 Distributed Shared Memory18 Mach Case Study
System infrastructure
Middleware
5 Distributed Objects and Remote Invocation 7 Security 9 Name Services17 CORBA Case Study
12 Transactions and Concurrency Control13 Distributed Transactions14 Replication
Shared data
*
8
Can you name some examples of Operating Systems?
9
Can you name some examples of Operating Systems?
…Linux WinXP Vista Unix FreeBSD Mac OSX2K Aegis Scout Hydra Mach SPINOS/2 Express Flux Hope SpringAntaresOS EOS LOS SQOS LittleOS TINOSPalmOS WinCE TinyOS…
10
What is an Operating System?
User interface to hardware (device driver)
Provides abstractions (processes, file system)
Resource manager (scheduler) Means of communication (networking) …
11
Can you name some examples of Distributed Systems?
12
Distributed Systems Examples
Client-server (e.g., NFS) The Internet The Web A sensor network DNS BitTorrent (peer to peer overlays) Datacenters
13
What is a Distributed System?
Definition of a Distributed System
A distributed system is a collection of independent computers that appears to its users as a single system.
A definition (Coulouris, et al)
System of networked computers that
communicate and coordinate their actions only by passing messages
concurrent execution of programs components fail independently of
one another
A definition (Lamport)
“You know you have a distributed system when the crash of a computer you’ve never heard of stops you from getting any work done.” inter-dependencies shared state
Definition of a Distributed System
A distributed system organized as middleware.Note that the middleware layer extends over multiple machines.
1.1
Distributed System
Distributed Systems 19
Next Generation Information Infrastructure
Visualization
BattlePlanning
Visualization
Collaborative Multimedia
(Telemedicine)
Collaborative Task Clients
Server farms
BattlePlanning
ElectronicCommerce Distance Learning
Requirements - Availability, Reliability, Quality-of-Service, Cost-effectiveness, Security
DeviceNets &
SensorNets
Wide Area Network (Internet)
Distributed Systems 20
Mobile & ubiquitous distributed systems
Distributed System Example – A Datacenter
Servers
Front Back
In Some highly secure (e.g., financial info)
Power
Off-site
On-site•WUE = Annual Water Usage / IT Equipment Energy (L/kWh) – low is good•PUE = Total facility Power / IT Equipment Power – low is good
(e.g., Google~1.11)
Cooling
Air sucked in from top (also, Bugzappers) Water purified
Water sprayed into air 15 motors per server bank
25
List of Topics we’ve Covered
Distributed system Basics Lamport timestamps File system Name space Failure detectors Replication Clouds and their predecessors (e.g., Grids and
timesharing industry) Sensor networks
Operating System types Centralized Systems
Process management Network System
Share resources Remote access Telnet / FTP No direct control from machine to another
Distributed system Global view of files system Global name Global time….
Example of Distributed Systems Network of workstations in university
It has a single file system with all files accessible from all machines
Workflow information system (automatic order) Orders are placed by means of laptop that are
connected to the system through telephone network
World Wide Web To publish a document by give it unique URL
Distributed Application Examples
Automated banking systems
Tracking roaming cellular phones
Air-traffic control
Example:Embedded Systems Automotive control systems
Mercedes S class carsthese days are equippedwith 50+ autonomous embedded processors
Connected throughproprietary bus-like LANs
Consumer electronics Audio HiFi equipment
Sensor Networks
Organizing a sensor network database, while storing and processing data (a) only at the operator’s site or …
Sensor Networks
Organizing a sensor network database, while storing and processing data … or (b) only at the sensors.
)A (Plants and Animals interacting in the Food Chain
Which is a Distributed System – (A) or (B)?
– Which is a Distributed System(A) or(B)?
(A)
)B (The Internet (Internet Mapping Project, color coded by ISPs)
(B)
34
What is a Cloud? It’s a cluster! It’s a supercomputer!
It’s a datastore! It’s superman!
None of the above All of the above
Cloud = Lots of storage + compute cycles nearby
35
A Sample Cloud Topology
Top of the Rack Switch
Core Switch
Servers
Rack
If higher bandwidth link ,then a “fat tree” topology
36
Timesharing Industry (1975):•Market Share: Honeywell 34%, IBM 15% ,•Xerox 10%, CDC 10%, DEC 10%, UNIVAC 10%•Honeywell 6000 & 635, IBM 370/168 ,
Xerox 940 & Sigma 9, DEC PDP-10, UNIVAC 1108
Grids (1980s-2000s):•GriPhyN (1970s-80s)•Open Science Grid and Lambda Rail (2000s)•Globus & other standards (1990s-2000s)
First large datacenters: ENIAC, ORDVAC, ILLIACMany used vacuum tubes and mechanical relays
P2P Systems (90s-00s)•Many Millions of users•Many GB per day
Data Processing Industry - 1968 :$70 M. 1978: $3.15 Billion.
Berkeley NOW ProjectSupercomputers
Server Farms (e.g., Oceano)
“A Cloudy History of Time” © IG 2010
Clouds
37
Course Projects
1. Multiparty Democracy and distributed systems2. Smartphones for healthy living3. Mobile ad-hoc live streaming4. Social network5. Datacenter topology
6. New transport protocol7. Graph analysis8. Energy-efficient datacenters9. DNS10. VM scheduler11. Instant Messaging12. New cloud services13. Pricing for clouds14. Health information systems