Networks and Distributed Systemsa.k.a. G22.3033-010

Lakshmi Subramanianhttp://cs.nyu.edu/~lakshmi

Jinyang Lihttp://cs.nyu.edu/~jinyang

Class goals

• Help you – critically appreciate networks & systems research– learn creative problem solving (i.e. doing research)

• How?– Lectures/readings: discuss state-of-art work– Programming labs: play with real systems– A semester-long research project

Syllabus, grading etc.

• http://www.cs.nyu.edu/courses/fall06/G22.3033-010

• Class participation (20%)– Read assigned papers before class!

• Two labs (10%)• One project (70%)

– Team of 2-3 people (<= 1 Ph.D. student per group)– Start next week– Weekly (or once every two weeks) meetings

Who should take the class?

• Grad-level class– Satisfy M.S. requirement of a “project” course

• Pre-requisite:– Basic knowledge on networks

• Computer Networks (L. Peterson)• An engineering approach to computer networking

(S. Keshav)

– Programming experience• TCP/IP Illustrated (R. Stevens)

Misc.

• Office hours:– Jinyang: 715 Broadway Rm 705, Tue 5-6pm– Lakshmi: Rm 706 Mon 5-6pm– TA: Ja Chen (jchen@cs.nyu.edu)

Next Generation Networks

Jinyang Li

Emerging networks

• Wireless networks

• Sensor networks

• Overlays and P2P

• Delay tolerant networks (DTNs)

• …

Wireless networks

Wireless networks: why now?• Proliferation of wifi-enabled devices• Faster, cheaper radios and more powerful boxes

Wireless apps: urban mesh

• Provide cheap, ubiquitous Internet connectivity

MIT Cambridge Roofnethttp://pdos.lcs.mit.edu/roofnet

Google Mountain View pole top networkhttp://wifi.google.com

Wireless apps: connecting rural villages

Intel/UC Berkeley/NYU Tier projecthttp://tier.cs.berkeley.edu

Wireless apps: mobile, ad-hoc communication

MIT CarTelhttp://cartel.csail.mit.edu

Wireless networks: challenges

1. Crappy links

2. Contention and self-interference

3. Frequent node/link failures

4. Many parameters

Goal: Robust, high performance designs• MAC layer• Routing layer• Transport layer

Challenge #1: crappy links

• Many asymmetric, lossy links

Challenge #2: contention• Many nodes access the medium collisions• No way to explicitly detect collisions

Challenge #2: self-interference

• A multi-hop flow interferes at successive hops

1 2 3 4 5

• At most every third node can transmit

Challenges #3: dynamism

• Links/nodes fail and recover frequently

• Link qualities change over time

Time (sec)

Challenge #4: (too) many tunable parameters

• Transmission power

• Transmission rate

• Directional vs. omni antennas

• Static vs. dynamic channel assignment

• One vs. multiple radios

Current state-of-art

MIT Roofnet pair-wise node throughput (11Mbps 802.11b radios)

# hops latency

(ms)

throughput

(kbps)

1 14 2451

2 26 771

3 45 362

4 50 266

5 60 210

6 100 272

7 83 181

Sensor networks

Beyond host-to-host communication

Sensor networks: why now?• Technology is ready

– Cheaper, smaller, more powerful sensors– Sense light, temperature, vibration, humidity, location, pulse, motion, vital sign etc.

• Monitor environment, collection information

UCB Telos Xbow MicaZIntel Dot

Sensor apps: understanding redwood forests

UC Berkeley/Intel Research

Sensor apps: real-time patient tracking

HarvardCodeBlue

Sensor-net challenges

• Different communication paradigm– host-to-host is the wrong fit– Data-centric

• Limited resources– Low radio bandwidth

250Kbps advertised, ~80Kbps in real life

– Slow processor, tiny storage8MHz CPU, 8K RAM

– Limited energy

Overlays and P2P

Distributed systems meet the Internet

Why p2p/overlay?

• A distributed system architecture:– No (minimal) centralized control– Nodes are symmetric in function

• Enabled by technology improvements

Internet

Large scale wide-area systems• Unmanaged (open p2p systems):

– BitTorrent: >1M nodes– Skype: >5M users

• Managed – PlanetLab: 700 nodes over 336 sites– Akamai CDN: >10K nodes

What’s new here?

• Opportunities:– Huge aggregate capacity

Network, storage, processing…

– Geographic diversity

• Many apps: – File sharing– CDNs– VoIP– Streaming multicast– Usenet news– …

Challenges• How to find data?• How to deal with failures?

– Nodes fail and recover– Network outage and partition

• (Open networks only) How to deal with selfish or malicious nodes?

– provide data integrity– provide privacy or anonymity

Challenge #1: resource discoveryCase study: file sharing

• Where is the file named “Hamlet”?

Challenge #2: churn

• What if the node with “Hamlet” goes down?

Challenge #3: selfish nodes

• Selfish nodes do not want to upload “Hamlet”

I do NOT haveHamlet

Challenge #4: malicious nodes

I HAVE junk named Hamlet

• Malicious nodes lie about their contents

Next week

• Naming and addressing• Project ideas

Check out the Spring class“distributed storage systems”

Distributed systems in a data-center

• Connected by LANs low loss and delay

• Provide infrastructural services for apps – Network file systems– Databases– Distributed data processing