CS 351/ IT 351 Modeling and Simulation Technologies
description
Transcript of CS 351/ IT 351 Modeling and Simulation Technologies
CS 351/ IT 351 Modeling and Simulation
Technologies
Errors In Models
Dr. Jim Holten
CS 351/ IT 351
Errors in Models
• Sources of Errors
• Characterizing Errors
• Using Error Bounds
• Interpreting Error Implications
CS 351/ IT 351
Sources of Errors
• Input Values (measurements)
• Machine Inaccuracies
• Algorithm Inaccuracies
• Bad models
CS 351/ IT 351
Measurement Errors
• Measurement granularity
• Granularity accuracy ==> Error intervals
• Types of measurements
CS 351/ IT 351
Machine Errors: Representation
• Float: 7 decimal places, E+/-38, or
subnormal E-45 (fewer digits of precision)
• Double – 16 decimal places, E +/-308, or
subnormal E-324 (fewer digits of precision)
CS 351/ IT 351
Machine Errors: Representation
• Equality comparisons (does 0.0F == 0.0D?)
• Overflow (too big an exponent)
• Underflow (too small an exponent)
• Mismatch (1.000E19D + 47.3D = ?)
CS 351/ IT 351
Machine Errors
• Divide by zero (+/- Inf), or divide zero by zero (NaN)
• Propagate “bad” values
• Worst-case scenarios, not seen as errors
– Near zero results of add or subtract
– Near zero denominator
CS 351/ IT 351
Algorithm Sources of Errors
• Inaccurate representation of real world
• Inaccurate representation of ideal world
• Computational errors
CS 351/ IT 351
Real World to Ideal Model
• Math Models are Idealistic
• Real world has many perturbations
• Statistical estimates are only “best fit” to
observed measurements
• Results in an inaccurate ideal model
CS 351/ IT 351
Ideal Model to Implementation
• Machine errors in number representations
• Machine errors in arithmetic calculations
• Results in even worse implementation
model values
CS 351/ IT 351
Computational Errors
• Numerical calculation to approximate math
functions
• Numerical Integration
• Numerical differentiation
• Techniques used determine the error
behaviors
CS 351/ IT 351
Controllable Errors• Understanding sources and behavior of
errors empowers you to control them and
predict their effects on the results.
• Identifying sources and effects of errors
allows you to better judge the quality of
models.
CS 351/ IT 351
What Gives Bad Models?
• Wrong equations
• Wrong numerical methods
• Details gone awry
• All irrationally affect results.
CS 351/ IT 351
Characterizing Errors
• Error Forms (Probability Distributions?)
• Error propagation effects on error forms
• Limitations versus needs
CS 351/ IT 351
Error Characterizationss• Error probability distributions
• The normal distribution• Zoo of common other distributions• Arbitrary distributions
• Error bounds
• Generalized error estimation functions
• Enumerated values and “false negatives”
CS 351/ IT 351
Error Probability Distributions
• Measurement error characteristics
• Calculation error characteristics
• Introduced algorithmic error terms
CS 351/ IT 351
Measurement ErrorCharacteristics
• Discrete sample on a number line
• Spacing determines “range” for each
measurement point
• Actual value may be anywhere in that range
CS 351/ IT 351
Calculation ErrorCharacteristics
• Round-off
• Divide by near-zero
• Divide by zero
• Algorithm inaccuracies
CS 351/ IT 351
Algorithmic ErrorCharacteristics
• Depends on the algorithms/solvers used
• Depends on the problem size
• Depends on inter-submodel data sharing
patterns and volume
CS 351/ IT 351
Errors: Normal Distributions
• Easy to characterize
• Propagates nicely through linear stages
• Useless for nonlinearities, special
conditions
• Not always a good fit
CS 351/ IT 351
Errors:Generalized Distributions
• Not commonly used
• Easy to represent (histograms into PDFs)
• Propagate through nonlinear calculations?
• Awkward: histograms, PDFs, CDFs for
each variable
CS 351/ IT 351
Errors: Bounded
• Not commonly used
• Easy to represent (+/-error magnitude)
• Can be propagated through nonlinear
calculations
• Still awkward for some calculations
CS 351/ IT 351
Errors: Propagating a Distribution
• Highly dependent on the distribution and
the calculations being performed.
• Generally only linear operations give easily
predictable algebraic results.
• Others require piecewise approximations
CS 351/ IT 351
Error Bounds• Expected value, +/-error magnitude, or min/max
• Propagates through calculations?
• More complex forms may be needed after
propagation – bounded piecewise linear
distributions
CS 351/ IT 351
Errors: Unhandled Implications
• Misinterpretation of results
• Misplaced confidences
• “Chicken Little”, “The Boy Who Cried 'Wolf'”, and ignored real consequences