Evaluating the Benefit of Networked EW Systems Greg Chalmers and Keith Mason.
Evaluating Networked Systemscs5480/notes/netclass.pdf · 2007. 9. 18. · Evaluating Networked...
Transcript of Evaluating Networked Systemscs5480/notes/netclass.pdf · 2007. 9. 18. · Evaluating Networked...
Evaluating Networked Systems
Robert Ricci September 2007
Network Evaluation
Network EvaluationSo, you've built a faster BitTorrent
Or built a new video-on-demandservice
Or improved TCP
But how can you be sure it's better?
And how can you convince others?
3
Network EvaluationMethodologies
Ad-hoc experimentation
Live-network experimentation
Simulation
Emulation
4
Ad-hoc Experimentation
5
Ad-hoc ExperimentationJust try it on some machines on aLAN
... but are the network conditionsbelievable?... and is it big enough?
Plus, you might need to do a lot ofwork to get it set up
And it wouldn't be very automated
6
Live Internet Experimentation
7
Live Internet ExperimentationGet access to machines sitting around theInternet
PlanetLab is a popular way to do this
Much more believable network conditions
... but network conditions are notcontrollable... and are not repeatable... and it can be hard to tell what's goingon out there
You might be able to get real users
8
PlanetLabLinux machines around the world
Over 300 working ones at over 160sitesShared among many usersMost at universities
People run real services on it
Coral - .nyud.net URLsCoDEEN content distribution
www.planet-lab.org9
Simulation
10
SimulationTotally different tactic - don't run on a realnetwork at all!
Simulate the behavior of yourapplication/protocol
Application-specific simulatorsGeneral network simulators (like NS)
Totally repeatable, controllable, andtransparent
... but you can miss out on a lot of things bynot running a real application, OS... and most developers would rather workwith something "real"
11
NS-2, the Network SimulatorPopular general-purpose networksimulator
Great for detailed examination ofprotocols
Several TCP (and other protocol)implementations
Traffic generators
NAM for visualizing network activity
12
Emulation
13
EmulationCombine the previous three strategies:
Run real application on a bunch ofmachines all in one placeBut emulate realistic networkcharacteristics between themProviding a controllable, repeatable,and transparent environment
Still not perfect, though
Network emulation is only as good asthe conditions you tell it to emulate
14
Emulab
What is Emulab?An emulator
... and a lot more
A full-service experimentation platform
Experiment management, collaboration tools,etc.
A lot of hardware
Over 360 PCs, over in the MEB
A lot of code
Software runs two dozen testbeds
Created here at the U by the Flux Group
16
Emulab Hardware - Front
17
Emulab Hardware - Back
18
Emulab BasicsYou describe a network
Network links and LANs, including BW,latency, etc.Host characteristics, including OSSoftware to install
Emulab builds your network for you
You get exclusive access to some PCsYou get an isolated network with thecharacteristics you asked forWithin a few minutes
19
What Do People Use It For?Evaluation of new/improved network protocols
Peer-to-PeerDistributed SystemsMulticast
Security Research
IDSThreat Response
Reliability Work
Operating System Research
Classes
20
Experimentation on Emulab
Demo Building a Topology With GUI
How To Design A GoodExperiment
Use the scientific method
Test hypothesis"BitDeluge is faster than BitTorrent underpacket loss"
Change only one thing at a timeJust client bandwidth, or packet loss, not both]
When comparing to an existing system, makesure to compare apples to apples
Automate as much as possible
Collect as much data as you can, and sanitycheck
23
Running Your ExperimentInteractive: Log in via SSH
Remember, it's "just a PC"Good for development/debugging
Scripted: Use "event system"
Can fire off programs, change networkconditionsBrings consistency and repeatability toyour experimentLet experiments run while you sleep
24
Demo Logging in And Running Simple Commands
How Emulab Builds Your Network
Setting Up Your NodesLoads disks with custom diskloader (Frisbee)
You get the nodes to yourself
You get full root access
Shared network-mounted homedirectory
Nodes get wiped when you'redone
27
Setting Up Your NetworkVLANs provide the links and LANs
Isolated form other experiments
Automatically configures IPaddresses and routing
Inserts delay nodes for trafficshaping
28
VLANs and IsolationVLAN: Virtual LAN
Switch technology that limitswhich ports can communicate
Provides the illusion of manyexclusive-use switches
Emulab switches provisioned toavoid performance artifacts
29
Emulating NetworkCharacteristics: Delay Nodes
Transparently inserted in your linksand LANs
'Bridge' packets between interfacesat Layer 2
Can be brought down to emulate linkfailure
Passes packets through dummynet todo traffic shaping
30
dummynet details
Packetsenter
Packetsleave
BandwidthQueue
DelayQueue
AvailablebandwidthPacket
Dropper
Bandwidth Queue: Bandwidth,Queuing and Transmission Delay
Delay Queue: Propagation Delay
Packet Dropper: Packet Loss31
The Future: GENI
GENINSF-Funded "Global Environment for NetworkInnovations"
Facility for designing the next generationInternet
Enable experimentation at the network layer
Improves on the Live Internet method bycontrolling its own backbone
Virtualization allows multiple networkarchitectures on the same substrate
Some architecture work, prototype being donehere at the U
33
Wrap Up
Network EvaluationMethodologies
Ad-hoc experimentation
Useful for development, not serious evaluation
Live-network experimentation
Real, but unpredictable
Simulation
Good for very detailed studies, not so much forreal applications
Emulation
Good balance - reasonably realistic, goodcontrol/repeatability
35
EmulabPlatform for simulation, liveexperimentation, emulation
But mostly emulation
Created here at the U
Available to students andresearchers
Apply for an account atwww.emulab.net
36
end