ns-3: History and Future

ns-3: History and Future

Mathieu Lacage

March, 24th 2012

Lacage ns-3: History and Future March 2012 1 / 27

Outline

Introduction

Packet

Object Framework

Conclusion


How the project came to be

Multiple partners:I NSF-funded: UWA, GT, ISII INRIA-funded: planète

Pulling the project in multiple directions:

Wireless

Long Term

Survival

Fast TCP

Real Code

Integration

EmulationEducation

ns-3


How the project came to beMultiple partners:

I NSF-funded: UWA, GT, ISII INRIA-funded: planète


Wireless

Long Term

Survival

Fast TCP

Real Code

Integration

EmulationEducation

ns-3



I NSF-funded: UWA, GT, ISI

I INRIA-funded: planètePulling the project in multiple directions:

Wireless

Long Term

Survival

Fast TCP

Real Code

Integration

EmulationEducation

ns-3



I NSF-funded: UWA, GT, ISII INRIA-funded: planète


Wireless

Long Term

Survival

Fast TCP

Real Code

Integration

EmulationEducation

ns-3


Requirements

Real-code integration, Emulation:I Models must not be too abstract (Node, etc.)I Must be transparent to models

Fast TCP:I Memory efficiency of many packets in flight


Requirements

Real-code integration, Emulation:

I Models must not be too abstract (Node, etc.)I Must be transparent to models



Requirements

Real-code integration, Emulation:I Models must not be too abstract (Node, etc.)

I Must be transparent to modelsFast TCP:

I Memory efficiency of many packets in flight


Requirements




Requirements


Fast TCP:

I Memory efficiency of many packets in flight


Requirements




More requirements

Education:I Must be easy to use for students

Wireless:I Must have great validated wireless models

Long term survival:I Maintenance: production-quality codeI Funding: NSF & INRIA funding finite


More requirements

Education:

I Must be easy to use for studentsWireless:

I Must have great validated wireless modelsLong term survival:

I Maintenance: production-quality codeI Funding: NSF & INRIA funding finite


More requirements





More requirements


Wireless:

I Must have great validated wireless modelsLong term survival:



More requirements





More requirements



Long term survival:



More requirements



Long term survival:I Maintenance: production-quality code

I Funding: NSF & INRIA funding finite


More requirements





Long Term Survival

Hard to make enough money to maintain the code:I Small market (network researchers)I Large and complex code

Hard to obtain funding from research agencies:I Funds to use the toolI Funds to extend the toolI No funds to maintain the tool


Long Term Survival

Hard to make enough money to maintain the code:

I Small market (network researchers)I Large and complex code



Long Term Survival

Hard to make enough money to maintain the code:I Small market (network researchers)

I Large and complex codeHard to obtain funding from research agencies:

I Funds to use the toolI Funds to extend the toolI No funds to maintain the tool


Long Term Survival




Long Term Survival


Hard to obtain funding from research agencies:

I Funds to use the toolI Funds to extend the toolI No funds to maintain the tool


Long Term Survival


Hard to obtain funding from research agencies:I Funds to use the tool

I Funds to extend the toolI No funds to maintain the tool


Long Term Survival


Hard to obtain funding from research agencies:I Funds to use the toolI Funds to extend the tool

I No funds to maintain the tool


Long Term Survival




Open Source Community

Hard to create:I Licensing is trickyI MeritocracyI No discussions behind closed doorsI Must lose control to see the baby grow

Classic management problem:I Spend time to do things yourselfI Spend time to help others do the work

It is a hard pill to (make upper management) swallow



Hard to create:

I Licensing is trickyI MeritocracyI No discussions behind closed doorsI Must lose control to see the baby grow





Hard to create:I Licensing is tricky

I MeritocracyI No discussions behind closed doorsI Must lose control to see the baby grow





Hard to create:I Licensing is trickyI Meritocracy

I No discussions behind closed doorsI Must lose control to see the baby grow





Hard to create:I Licensing is trickyI MeritocracyI No discussions behind closed doors

I Must lose control to see the baby growClassic management problem:

I Spend time to do things yourselfI Spend time to help others do the work





Classic management problem:

I Spend time to do things yourselfI Spend time to help others do the work





Classic management problem:I Spend time to do things yourself

I Spend time to help others do the workIt is a hard pill to (make upper management) swallow


But the good news is...

We (They) all swallowed it.


And now,

The story of how:

I The Real World pervades everythingI To implement everything 3 or 4 times

I Aka, Incremental development

I To not plan too far


And now,

The story of how:I The Real World pervades everything

I To implement everything 3 or 4 timesI Aka, Incremental development



And now,

The story of how:I The Real World pervades everythingI To implement everything 3 or 4 times

I Aka, Incremental development



Outline

Introduction

Packet

Object Framework

Conclusion


Requirements

I Transparent conversion to/from real bytesI CPU and memory efficiencyI Fragmentation/ReassemblyI Simulation-only dataI Pretty printingI ExtensibilityI Robust Application Programming Interface (API)


Requirements

I Transparent conversion to/from real bytes

I CPU and memory efficiencyI Fragmentation/ReassemblyI Simulation-only dataI Pretty printingI ExtensibilityI Robust Application Programming Interface (API)


Requirements

I Transparent conversion to/from real bytesI CPU and memory efficiency

I Fragmentation/ReassemblyI Simulation-only dataI Pretty printingI ExtensibilityI Robust Application Programming Interface (API)


Requirements

I Transparent conversion to/from real bytesI CPU and memory efficiencyI Fragmentation/Reassembly

I Simulation-only dataI Pretty printingI ExtensibilityI Robust Application Programming Interface (API)


Requirements

I Transparent conversion to/from real bytesI CPU and memory efficiencyI Fragmentation/ReassemblyI Simulation-only data

I Pretty printingI ExtensibilityI Robust Application Programming Interface (API)


Requirements

I Transparent conversion to/from real bytesI CPU and memory efficiencyI Fragmentation/ReassemblyI Simulation-only dataI Pretty printing

I ExtensibilityI Robust Application Programming Interface (API)


Requirements

I Transparent conversion to/from real bytesI CPU and memory efficiencyI Fragmentation/ReassemblyI Simulation-only dataI Pretty printingI Extensibility

I Robust Application Programming Interface (API)


Requirements

I Transparent conversion to/from real bytesI CPU and memory efficiencyI Fragmentation/ReassemblyI Simulation-only dataI Pretty printingI ExtensibilityI Robust Application Programming Interface (API)


Related Work

Two approaches

I Packet is list of headers: GTNetS, OMNeT++,SSFNet

TCPIP PayloadMAC

I Packet is buffer of bytes: Yans, GloMoSim

TCPIP PayloadMAC


Related Work

Two approachesI Packet is list of headers: GTNetS, OMNeT++,

SSFNet

TCPIP PayloadMAC

I Packet is buffer of bytes: Yans, GloMoSim

TCPIP PayloadMAC


Pros and Cons

List BufferFragmentation, Reassembly XConversion real bytes XSimulation-only data XPretty printing XCPU, memory efficiency X


Pros and Cons

List Buffer ns-3Fragmentation, Reassembly X XConversion real bytes X XSimulation-only data X XPretty printing X XCPU, memory efficiency X X


How did we get there ?

About 10 successive versions...



Started with a simple version in YANS:

Headers’ List

Tags

RefCounting

Sep

2005



Started with a simple version in YANS:

Headers’ List

Tags

RefCounting

Sep

2005

But, hard to integrate the BSD TCP stack



Real bytes in YANS:

Headers’ List

Tags

RefCounting

Real

Bytes

Sep

2005

Jan

2006



Real bytes in YANS:

Headers’ List

Tags

RefCounting

Real

Bytes

Sep

2005

Jan

2006

But, hard to manage reference counting by hand



Smart Pointers in YANS:

Headers’ List

Tags

RefCounting

Smart

Pointers

Real

Bytes

Sep

2005

Jan

2006

Jun

2006



Smart Pointers in YANS:

Headers’ List

Tags

RefCounting

Smart

Pointers

Real

Bytes

Sep

2005

Jan

2006

Jun

2006

But, memory & CPU performance is not good



Copy On Write in ns-3:

Headers’ List

Tags

RefCounting

Smart

Pointers

Real

BytesCOW

Sep

2005

Jan

2006

Jun

2006

Jul

2006



Copy On Write in ns-3:

Headers’ List

Tags

RefCounting

Smart

Pointers

Real

BytesCOW

Sep

2005

Jan

2006

Jun

2006

Jul

2006

Still, Fast TCP pushes the boundary



Virtual payload:

Headers’ List

Tags

RefCounting

Smart

Pointers

Real

BytesCOW

Virtual

Payload

Sep

2005

Jan

2006

Jun

2006

Jul

2006

Aug

2006



Virtual payload:

Headers’ List

Tags

RefCounting

Smart

Pointers

Real

BytesCOW

Virtual

Payload

Sep

2005

Jan

2006

Jun

2006

Jul

2006

Aug

2006

Really, debugging is hard



Packet UIDs:

Headers’ List

Tags

RefCounting

Smart

Pointers

Real

BytesCOW

Virtual

Payload

Packet

UID

Sep

2005

Jan

2006

Jun

2006

Jul

2006

Aug

2006

Sep

2006



Packet UIDs:

Headers’ List

Tags

RefCounting

Smart

Pointers

Real

BytesCOW

Virtual

Payload

Packet

UID

Sep

2005

Jan

2006

Jun

2006

Jul

2006

Aug

2006

Sep

2006

Really, debugging is still hard



Packet Pretty Printing:

Headers’ List

Tags

RefCounting

Smart

Pointers

Real

BytesCOW

Virtual

Payload

Pretty

Printing

Packet

UID

Sep

2005

Jan

2006

Jun

2006

Jul

2006

Aug

2006

Sep

2006

May/Jun

2007



Packet Pretty Printing:

Headers’ List

Tags

RefCounting

Smart

Pointers

Real

BytesCOW

Virtual

Payload

Pretty

Printing

Packet

UID

Sep

2005

Jan

2006

Jun

2006

Jul

2006

Aug

2006

Sep

2006

May/Jun

2007

But, we do need pointers



Pointers are back:

Headers’ List

Tags

RefCounting

Smart

Pointers

Real

BytesCOW

Virtual

Payload

Pretty

Printing

Packet

UIDPtr<Packet>

Sep

2005

Jan

2006

Jun

2006

Jul

2006

Aug

2006

Sep

2006

May/Jun

2007

Oct

2007


What I learned the hard way

I The mountain seems impossibly high at firstI Forget about the top, aim for base campI The mountain grows with every new userI Listen to users



I The mountain seems impossibly high at first

I Forget about the top, aim for base campI The mountain grows with every new userI Listen to users



I The mountain seems impossibly high at firstI Forget about the top, aim for base camp

I The mountain grows with every new userI Listen to users



I The mountain seems impossibly high at firstI Forget about the top, aim for base campI The mountain grows with every new user

I Listen to users



I The mountain seems impossibly high at firstI Forget about the top, aim for base campI The mountain grows with every new userI Listen to users


Outline

Introduction

Packet

Object Framework

Conclusion


Requirements

I Uniform memory managementI Allow simulation configurationI Solve the object composition problemI Decouple Trace Sources from Trace SinksI Allow simulation trace setupI Do not be intrusive to Model developers


Requirements

I Uniform memory management

I Allow simulation configurationI Solve the object composition problemI Decouple Trace Sources from Trace SinksI Allow simulation trace setupI Do not be intrusive to Model developers


Requirements

I Uniform memory managementI Allow simulation configuration

I Solve the object composition problemI Decouple Trace Sources from Trace SinksI Allow simulation trace setupI Do not be intrusive to Model developers


Requirements

I Uniform memory managementI Allow simulation configurationI Solve the object composition problem

I Decouple Trace Sources from Trace SinksI Allow simulation trace setupI Do not be intrusive to Model developers


Requirements

I Uniform memory managementI Allow simulation configurationI Solve the object composition problemI Decouple Trace Sources from Trace Sinks

I Allow simulation trace setupI Do not be intrusive to Model developers


Requirements

I Uniform memory managementI Allow simulation configurationI Solve the object composition problemI Decouple Trace Sources from Trace SinksI Allow simulation trace setup

I Do not be intrusive to Model developers


Requirements

I Uniform memory managementI Allow simulation configurationI Solve the object composition problemI Decouple Trace Sources from Trace SinksI Allow simulation trace setupI Do not be intrusive to Model developers


Fast Forward to 2012

I Memory management: Ptr<Object> + DisposeI Configuration: Attributes + ConfigStoreI Tracing: integrated in AttributesI Configuration, Tracing: optional

Now on par with Omnetpp & Opnet feature-wise



I Memory management: Ptr<Object> + Dispose

I Configuration: Attributes + ConfigStoreI Tracing: integrated in AttributesI Configuration, Tracing: optional




I Memory management: Ptr<Object> + DisposeI Configuration: Attributes + ConfigStore

I Tracing: integrated in AttributesI Configuration, Tracing: optional




I Memory management: Ptr<Object> + DisposeI Configuration: Attributes + ConfigStoreI Tracing: integrated in Attributes

I Configuration, Tracing: optional




I Memory management: Ptr<Object> + DisposeI Configuration: Attributes + ConfigStoreI Tracing: integrated in AttributesI Configuration, Tracing: optional



A Windy Path

It Started with nothing


A Windy Path

But tom liked the ns-2 default values

Default

Value

Mar

2007


A Windy Path

But tom liked the ns-2 default values

Default

Value

Mar

2007

What about memory management, aggregation ?


A Windy Path

Here comes microsoft’s IUnknown!

Default

Value

IUnknown

Mar

2007

Apr

2007


A Windy Path

Here comes microsoft’s IUnknown!

Default

Value

IUnknown

Mar

2007

Apr

2007

Scratch that, everyone hates it


A Windy Path

Resolving trace sources: microsoft’s IMoniker

Default

ValueIMoniker

IUnknown

Mar

2007

Apr

2007


A Windy Path

Resolving trace sources: microsoft’s IMoniker

Default

ValueIMoniker

IUnknown

Mar

2007

Apr

2007

Hrm... So, what about memory management ?


A Windy Path

Reference Counting wins

Default

Value

Reference

Counting

IMoniker

IUnknown

Mar

2007

Apr

2007

May

2007


A Windy Path

Reference Counting wins

Default

Value

Reference

Counting

IMoniker

IUnknown

Mar

2007

Apr

2007

May

2007

And Cycles ?


A Windy Path

Dispose to Break the Cycles

Default

ValueDispose

Reference

Counting

IMoniker

IUnknown

Mar

2007

Apr

2007

May

2007


A Windy Path

Dispose to Break the Cycles

Default

ValueDispose

Reference

Counting

IMoniker

IUnknown

Mar

2007

Apr

2007

May

2007

Where is my IUnknown::QueryInterface ?


A Windy Path

Dynamic Aggregation

Default

ValueDispose

Reference

CountingAggregation

IMoniker

IUnknown

Mar

2007

Apr

2007

May

2007


A Windy Path

Dynamic Aggregation

Default

ValueDispose

Reference

CountingAggregation

IMoniker

IUnknown

Mar

2007

Apr

2007

May

2007

Ref/Unref by hand still sucks!


A Windy Path

Smart Pointers are back

Default

ValueDispose

Reference

Counting

Ptr<>

Aggregation

IMoniker

IUnknown

Mar

2007

Apr

2007

May

2007


A Windy Path

Smart Pointers are back

Default

ValueDispose

Reference

Counting

Ptr<>

Aggregation

IMoniker

IUnknown

Mar

2007

Apr

2007

May

2007

IMoniker is still ugly...


A Windy Path

Introduce Attributes

Default

ValueDispose

Reference

Counting

Ptr<>

Aggregation

IMoniker Attributes

IUnknown

Mar

2007

Apr

2007

Jan-Mar

2008

May

2007


With hindsight...

I wish:I we had looked at Omnetpp & Opnet before

The result is still nice thanks to:I Willingness to listen to usersI Incremental development


With hindsight...


The result is still nice thanks to:I Willingness to listen to users

I Incremental development


With hindsight...


The result is still nice thanks to:I Willingness to listen to usersI Incremental development


No, it is not over

Real Code Integration:I Linux kernel, Quagga, CCNx, etc.I ContikiI TinyOS

Brings new challenges:I Attributes ?I Tracing ?


No, it is not over

Real Code Integration:I Linux kernel, Quagga, CCNx, etc.

I ContikiI TinyOS



No, it is not over

Real Code Integration:I Linux kernel, Quagga, CCNx, etc.I Contiki

I TinyOSBrings new challenges:

I Attributes ?I Tracing ?


No, it is not over




No, it is not over


Brings new challenges:I Attributes ?

I Tracing ?


No, it is not over




Configuring Attributes

Linux Kernel:I Bridge /proc/net inside the attribute systemI Difficult to really map one to oneI Extend Attributes to handle structures, arrays ?

Other Applications:I I have no idea.



Linux Kernel:

I Bridge /proc/net inside the attribute systemI Difficult to really map one to oneI Extend Attributes to handle structures, arrays ?




Linux Kernel:I Bridge /proc/net inside the attribute system

I Difficult to really map one to oneI Extend Attributes to handle structures, arrays ?




Linux Kernel:I Bridge /proc/net inside the attribute systemI Difficult to really map one to one

I Extend Attributes to handle structures, arrays ?Other Applications:

I I have no idea.




Other Applications:

I I have no idea.


Tracing

We do not want to modify the external code:I Linux Kernel: Tracepoints, Ftrace, KProbe, etc.I But no builtin tracing in general

Re-implement Kprobe, JProbe, etc. in userspace:I Introspection using debug informationI Dynamic patching of assembly code


Tracing

We do not want to modify the external code:

I Linux Kernel: Tracepoints, Ftrace, KProbe, etc.I But no builtin tracing in general



Tracing

We do not want to modify the external code:I Linux Kernel: Tracepoints, Ftrace, KProbe, etc.

I But no builtin tracing in generalRe-implement Kprobe, JProbe, etc. in userspace:

I Introspection using debug informationI Dynamic patching of assembly code


Tracing




Tracing


Re-implement Kprobe, JProbe, etc. in userspace:

I Introspection using debug informationI Dynamic patching of assembly code


Tracing


Re-implement Kprobe, JProbe, etc. in userspace:I Introspection using debug information

I Dynamic patching of assembly code


Tracing




Outline

Introduction

Packet

Object Framework

Conclusion


Summary

What really matters (today):I Alignment with the Real WorldI An Open Source CommunityI Incremental developmentI No one (but you) knows where ns-3 is headed


Summary

What really matters (today):

I Alignment with the Real WorldI An Open Source CommunityI Incremental developmentI No one (but you) knows where ns-3 is headed


Summary

What really matters (today):I Alignment with the Real World

I An Open Source CommunityI Incremental developmentI No one (but you) knows where ns-3 is headed


Summary

What really matters (today):I Alignment with the Real WorldI An Open Source Community

I Incremental developmentI No one (but you) knows where ns-3 is headed


Summary

What really matters (today):I Alignment with the Real WorldI An Open Source CommunityI Incremental development

I No one (but you) knows where ns-3 is headed


Summary

What really matters (today):I Alignment with the Real WorldI An Open Source CommunityI Incremental developmentI No one (but you) knows where ns-3 is headed


Thank you !

Questions ?


Contact

[email protected]


ns-3: History and Future

Technology

Transcript of ns-3: History and Future