Performance Evaluation: 30 years in review
description
Transcript of Performance Evaluation: 30 years in review
Performance Evaluation: 30 years in reviewMartin ReiserGMD – German National Research Center for Information Technology
MMB’99, 22.–24.September 1999, Trier
Presentation 23. September, 9:00
Very complex systems… where are the limits?
… will it work?
… if so, how well?
… can it work better?
… what will I need tomorrow, in a month, year …?
but why bother anyway?
Host
Switch
Link
ALU
Memory
Channels
30 years?m
any
subs
crib
ers
m tr
unk
line
s
?…
A. K. Erlang, The Post Office Electrical Engineers's Journal 1918
80!
E1,m (ρ) =
ρm
m!ρk
k!k=0m∑
Cutting the Gordian knot
Kleinrock, 1964 Gaver, 1967; Buzen, 1971
Message Independence Central Server
Big bang: BCMP
i
j
closed class s
G(•)
open class r r
i
j
class change pi,r; j ,s
Ns Jobs
Baskett, Chandy, Muntz and Palacios, 1975
LCFS/PRPSIS
E( )
G( )
FCFS
unlimited waiting roomqueue-dependent rates
The PF solution is robust
P(n) C(nii
) Wi,rni ,r
r
station i, class r:
Routing details don’t matter, only mean number of visits does!
Distribution of service-times does not matter, only mean values enter into the solution!
long live the Central Server
The classical models
Delay analysis of data networks (Kleinrock 64)
Time-sharing model (Scherr 67)
Sharing the CPU, i.e. PS (Kleinrock et. al. 64 +)
Central Server (Buzen 71)
Polling, token-ring (Konheim et. al. 74 +)
Aloha, CSMA …(Kleinrock, Lam, Tobagi 73 +)
From trees
3274
3705
3274
370
3705
to meshed networks
From window flow-control
Ti (W) i 1 ni (W 1)
W
Ti (W)ni(W) Ti(W)
to MVA
From Poisson arrivalscall
to MAP…
1
silence talk spurt
2
MPX
20
40
50
60
70
30
Product-Form Qing Theory State-space
Moline (PAP) Erlang MM∞/K
Tandem
Jackson
Gordon/Newell
Convolution
BCMP
Norton's theorem
MM1
Crommelin MDk
MG1LittleCohen
Cyclic Service
Conservation LawsPS
Simon Ando
Courtois
Methods
80
90
MVA
BlockingExotic Qs
Waiting timesG-Networks
customized MVA
PF-Petri Nets
Matrix geom.
Complex tandem
MAP queues
Self-similar trafficLarge deviation th.
UniformizationPerformability
GSPN
Iterative Meth.
DSPNArnoldi
Algebraic meth.
Product-Form Qing Theory State-spaceMethods
Architecture Communication Operation
40
50
60
70
80
90
Capacity planning(BEST1)
Manufacturing
Software Perf. Eng.
Workflow
Netw. mgmt.
Time-sharingVirtual memory
Central Server
Processor arch.Memory hierarchy
(caches)Multiprocessor
Parallel pgm's
Mem. consistencyPrefetching
multithreaded arch.
Telephony
Traffic theory
Delay analysis
Links (Polling)
Flow control
LAN
ATMBroadband
Interconn. netw's
Selfsimilar traffic
Internet, WWW
30
Problems
40
50
60
70
80
90
Capacity planning(BEST1)
Manufacturing
Software Perf. Eng.
Workflow
Netw. mgmt.
Time-sharingVirtual memory
Central Server
Processor arch.Memory hierarchy
(caches)Multiprocessor
Parallel pgm's
Mem. consistencyPrefetching
multithreaded arch.
Telephony
Traffic theory
Delay analysis
Links (Polling)
Flow control
LAN
ATMBroadband
Interconn. netw's
Selfsimilar traffic
Internet, WWW
30
Architecture Communication OperationProblems
PE specialists… work on PE problems cached in PE terms
… build tools
… are integral parts of development teams
… develop (and solve) models of real systems
really?
… use PE methodology in their daily work (planning, managing, tuning)
IS professionals
Success stories
PE-based engineering and development
Network management
Capacity planning
Failures ?
Software performance engineering
PE-basierte F+ESkilled PE specialist group
– is integrated in R&D team– is of sufficient size (5 – 10%)– is highly respected– and contributes to design!
All important design decisions are basedon PE results and analysis
Early prototype supportPE predictions
Case study: IBM Token Ring
VLSI Design
Generally accepted specification process
Simulation data from specification • on several levels of detail
Automatic mask generation
Generation of test patterns
integrated processenvironment
PE in standards
Success story LAN standard IEEE 802
Success story ATM
fix before build
Early attempts in SDLC/HDLC
What is PE?
Is it a science
Is it used by engineers?
Is it used in IS planning/tuning?
Is it a business?
Is it regular part of university curriculum?
…………………………………………………yes
……………………………some
………………little
…………cottage industry
……not really
Quo vadis ?
Stay where we are?
Reach out to main stream?Must make quantum leap in SPE
Must integrate tool into develop-ment processes • robust • Sprache des Benutzers
• accepted in EE, especially communications
• continue existence as niche science
SPE roadmap
Rep.Usage
scenario editor
Workload data
Network capacityplanning
Analytic system structure model
System structure
editor
Capacityplanning
Path-lengthestimation
process
CASE me-thodology
In user’s language
3 grand challenges
Understand self-similar traffic • do we need a new theory? • se we merely shifting loads?
Introduce analytical methods in computerarchitecture (“a gold mine” S. Adve)
to enter into the main stream ofthe CASE process and tools