End-to-End Arguments in System Design

CS533 - Concepts of Operating Systems

End-to-End Arguments in System Design

Presentation by David Florey


Overview

The End-to-End Argument Considerations Guarantees Thoughts


End-to-End Argument

Simple File Transfer from Host A to Host Bo File can be corrupted in transito File can be corrupted when reado Buffered file can be corrupted in memory

Adding checks during transit may work, but what if they don’t work in all cases?

The only thing that is going to know if everything went down correctly is the highest level – the File Transfer Application

o Use checksums at Application Level


The End-to-End Argument

Host Computer A

Communication Subsystem

Host Computer B

File Transfer Application File Transfer Application

Communication Subsystem

Error checking, duplicate detection,etc

Level of reliability

Performance


ConsiderationsReliability and Performance

We cannot guarantee total reliability at the low level

o How much is enough? Performance tradeoffs

o Poor reliability due to little or no checking at low level leads to poor performance due to too many retries above

o Too much checking at low level leads to poor performance because of duplication of effort, perhaps unnecessary work

Middle ground – reliable enough, but not perfecto Low level should make its “best effort” to improve

performance without a complicated implementation


ConsiderationsPlacement of Things

Early checks can detect errors sooner, but is it the job of the application or of the subsystem?

o Consider what applications will be using the subsystem• Are they all going to benefit from this?• Should each on write their own check?• We are not adding to the low level to improve overall

reliability, we are adding to improve performance


Guarantees

Delivery Guaranteeso We only care if the receiver did the job or noto Subsystem should not try to make this guarantee

because only the application will know if the message made it or not

Secure transmissionso Responsibility of application: needs to check

authenticity and ensure that message is encrypted the entire time


Guarantees

Duplicate message suppressiono What’s your definition of a duplicate?o Details of an actual duplicate are unknown to

subsystem as this is very application specifico Therefore, since subsystem can’t guarantee to

suppress all types duplicate messages, and the app needs to detect these any way, the app should do all duplicate detection/supression


What is an “End”?

Consider the application o temporal accuracy vs logical accuracy

• Real-time or streaming voice or video applications can tolerate some amount of error at the application level

– Lossy compression algorithms do this– Perhaps I could say “Could you repeat that?”

• Low level error checking can get in the way and slow it down


Summary of End-to-End

Application knows besto Applications are better suited for error recovery,

duplicate suppression, encryption and so on Subsystems need to be reliable within reason,

but should not provide these “application” features


Thoughts

Larger conceptso “Open” Operating Systemso Applications know better how to manage resources:

• Microkernels• Scheduler activations• Exokernel

o Evolution of things• End-to-End seems to be the first step, the Neanderthals of

arguments• Better to consider solutions that can eventually be

standardized, automated and pushed to the next layer down or simply placed in a shared module


Thoughts

Parameterizationo Providing functionality at the low level like duplicate

suppression, while providing the ability to just turn it offo Consider potentially providing levels of functionality,

chosen by the application


Thoughts

This seems to be the main argument behind layering

o Functions layered, modularized

o FTP checks only FTP, TCP doesn’t care about it and IP doesn’t care about TCP

o No lower module performs jobs better suited in a higher module

FTP Client

TCP

IP

Ethernet Driver

FTP Client

TCP

IP

Ethernet DriverEther-Speak

IP-routing junk

TCP-stuff

FTP-talk


Conclusion

The End-to-End Argument should be considered for every low level function or for each “problem solved” at the low level

o Who’s going to use it?o What does it provide? Functionality? Performance?

Applications know best, but one should be aware of overheads and costs incurred when implementing functionality at ANY level

End-to-End Arguments in System Design

Documents

Transcript of End-to-End Arguments in System Design