A Comparative Analysis Of Contact Metrology Devices Versus Non-Contact Metrology
Systems Utilizing Structured Light
by
Michael Hestness
A Research Paper Submitted in Partial Fulfillment of the
Requirements for the Master of Science Degree
in
Manufacturing Engineering
The Graduate School
University of Wisconsin-Stout
December 2011
Copyright 2011 Lockheed Martin
2
Copyright 2011 Lockheed Martin
The Graduate School University of Wisconsin-Stout
Menomonie, WI 54751
Author: Hestness, Michael L.
Title: A Comparative Analysis of Contact Metrology Devices versus Non-
Contact Metrology Systems Utilizing Structured Light
Graduate Degree/ Major: Master of Science in Manufacturing Engineering
Research Adviser: John Dzissah
Month/Year: December, 2011
Number of Pages: 86
Style Manual Used: American Psychological Association, 6th
edition
Abstract
Modern aerospace manufacturing processes utilize the most advanced techniques
and technologies in the world. The need for lower cost and more efficient aircraft is
pushing the limits of current material and process capabilities. In an effort to meet these
requirements, aircraft designs require tighter tolerances than ever to ensure quality,
safety, and performance. While this may result in efficient airplanes with lower
operating costs, the initial costs to manufacture parts meeting stringent tolerances and
the subsequent inspection methods drive up production costs.
To meet these challenges, improved manufacturing and inspection techniques
are required. Traditionally, coordinate measuring machines (CMMs) and laser trackers
have been the metrology workhorses of aerospace. While these technologies are
accurate, they are slow and can be labor intensive to program and operate. Recent
3
Copyright 2011 Lockheed Martin
advances in non-contact inspection technologies such as white light, or structured light,
have produced impressive results but there is limited data to demonstrate these
technologies are as accurate as the legacy methods. This paper will compare the
capability of three white light systems to the capability of contact metrology
measurement devices by conducting a gage linearity and bias study and a gage
repeatability and reproducibility study for each white light scanning system.
4
Copyright 2011 Lockheed Martin
The Graduate School
University of Wisconsin Stout Menomonie, WI
Acknowledgments
I would like to thank my wife Amy and our kids for their support and patience
during my time in school. I would also like to thank the faculty of UW-Stout, especially
David Fly, for the support he and everyone has provided to me as a distance learning
student. Last but not least, I would like to thank Dr. John Dzissah for his time and
guidance completing this paper.
5
Copyright 2011 Lockheed Martin
Table of Contents
.................................................................................................................................. Page
Abstract ........................................................................................................................... 2
List of Tables ................................................................................................................... 7
List of Figures .................................................................................................................. 8
Chapter I: Introduction ..................................................................................................... 9
Statement of the Problem ................................................................................... 11
Purpose of the Study .......................................................................................... 12
Assumptions of the Study ................................................................................... 13
Limitations of the Study ...................................................................................... 13
Definition of Terms ............................................................................................. 14
Methodology ....................................................................................................... 15
Chapter II: Literature Review ......................................................................................... 17
Early Metrology Tools ......................................................................................... 17
Coordinate Measuring Machines ........................................................................ 19
Non-Contact Metrology ....................................................................................... 22
Gage Repeatability and Reproducibility .............................................................. 26
The Future of Non-contact Inspection ................................................................ 27
Chapter III: Methodology ............................................................................................... 28
Equipment .......................................................................................................... 28
Features and Test Articles .................................................................................. 32
Benchmarking ..................................................................................................... 35
6
Copyright 2011 Lockheed Martin
Non-contact Data Collection ............................................................................... 36
Gage Repeatability and Reproducibility Study ................................................... 44
Chapter IV: Results ....................................................................................................... 45
Results ............................................................................................................... 45
Summary of Findings .......................................................................................... 45 Gage Linearity and Bias Study ........................................................................... 46 Gage Repeatability and Reproducibility Study ................................................... 47
Chapter V: Discussion ................................................................................................... 50
Conclusions ........................................................................................................ 50
Recommendations .............................................................................................. 50
References .................................................................................................................... 52
Appendix A: Spot Face Panel Drawing ........................................................................ 55
Appendix B: Spot Face Panel Benchmark Results ....................................................... 56
Appendix C: Countersink Test Coupon Dimensions ..................................................... 57
Appendix D: Precision Surface Block ............................................................................ 58
Appendix E: Two-Ball, Ball Over Method Formula ........................................................ 59
Appendix F: Input Data and Results for Precision Surface Panel ................................. 60
Appendix G: Input Data and Results for Spot Face Panel ............................................. 69
Appendix H: Input Data and Results for Countersink Panel .......................................... 78
.
7
Copyright 2011 Lockheed Martin
List of Tables Table 1: Test Article Tolerances .................................................................................... 46 Table 2: Average Bias of Each Measurement System .................................................. 46 Table 3: Linearity Values for Each System Measuring and Each Test Article ............... 47 Table 4: %Linearity Values for Each System Measuring and Each Test Article ............ 47 Table 5: Gage R&R Repeatability Results .................................................................... 48 Table 6: Gage R&R Reproducibility Results .................................................................. 48 Table 7: Total Gage R&R Variation as A Percent of the Tolerance .............................. 48 Table 8: Acceptability of the Measurement Systems Based On 30% Variation/Tolerance
Ratio ................................................................................................................ 49
8
Copyright 2011 Lockheed Martin
List of Figures
Figure 1: Micrometer ..................................................................................................... 18 Figure 2: Vernier caliper ................................................................................................ 19
Figure 3: Zeiss Accura II AKTIV CMM ........................................................................... 29 Figure 4: CogniTens WLS 400M ................................................................................... 30 Figure 5: ATOS Triple Scan .......................................................................................... 31 Figure 6: Rexcan 4 White Light Scanner ....................................................................... 32 Figure 7: Precision Surface Block ................................................................................. 33 Figure 8: Spot Face Panel ............................................................................................. 34 Figure 9: Countersink Panel .......................................................................................... 35 Figure 10: Calibrating the WLS 400M ........................................................................... 37 Figure 11: Measuring the Precision Surface Block with the WLS 400M ........................ 38 Figure 12: Measuring the Spot Face Panel with the WLS 400M ................................... 38 Figure 13: Measuring the Countersink Panel with the WLS 400M ................................ 39 Figure 14: Calibrating the Rexcan 4 .............................................................................. 40 Figure 15: Mapping the Artifact with Photogrammetry................................................... 40 Figure 16: Measuring the Precision Surface Block with the Rexcan 4 ......................... 41 Figure 17: Measuring the Spot Face Panel with the Rexcan 4 ...................................... 41 Figure 18: Measuring the Countersink Panel with the Rexcan 4 ................................... 42 Figure 19: Measuring the Precision Surface Block with the ATOS Triple Scan ............. 42 Figure 20: Measuring the Spot Face Panel with the ATOS Triple Scan ........................ 43 Figure 21: Measuring the Countersink Panel with the ATOS Triple Scan ..................... 43
9
Copyright 2011 Lockheed Martin
Chapter I: Introduction
Many of today’s products have been designed with manufacturing tolerances that
were unachievable just ten years ago. As modern manufacturing processes improve
and become capable of producing these extremely precise parts, it becomes imperative
to have methods of inspection that are precise and capable of rapidly inspecting these
parts.
The automotive and aerospace industries have typically been the main users of
very precise and highly inspected parts. Components in an engine, whether it is an
automotive engine or a jet engine, are often held to tolerances in the ten thousandths of
an inch range. To measure these parts accurately, coordinate measuring machines
(CMMs) have long been considered a gold standard in metrology, which is the study of
measurements. A CMM is a measurement device used to make three-dimensional
measurements (Kalpakjian, 1992) and can come in several forms, but a typical bridge
CMM consists of a granite bed, a touch probe, and a gantry system allowing very
precise control of the touch probe in three axes.
As the aerospace industry develops larger aircraft with increasingly tight
tolerances to achieve the desired performance, the need to build larger CMMs becomes
clear. These new CMMs are very expensive because as the inspection volume
increases, so does the cost. Many CMMs used in aerospace are housed in
environmentally controlled buildings. Larger CMMs require larger buildings and more
controls to maintain a constant room temperature. Also, as the machines become
larger they need to be made more rigid and sit on sturdier foundations.
10
Copyright 2011 Lockheed Martin
A long Y-axis bridge is more susceptible to bending and vibrations than a short
bridge. In addition, the further the CMM travels from its home or start position (x=0,
y=0, z=0), the more likely it is to lose accuracy. To compensate for this, large CMMs
need better encoders and specialized software to ensure they remain accurate over a
large volume.
Lockheed Martin is producing the F-35 Joint Strike Fighter and is expected to
produce one plane per day when full rate production is realized. As the only 5th
generation fighter jet in production, no other airplane in the world is manufactured to the
stringent tolerances required to ensure the plane meets its performance goals. A 5th
generation plane is considered a low observable (LO) plane and is more commonly
known as a stealth plane. LO planes are designed to be nearly invisible to radar. To
achieve LO performance, the plane utilizes unique shape characteristics, advanced
materials, and special coatings. All of these must be strictly controlled and verified.
The use of the digital thread (Kinard, 2010) has also added to the challenges of
aerospace manufacturing. Coined by Lockheed Martin, the term digital thread means
there are no master tools used to build parts. Every part produced is controlled by a
three-dimensional computer model. Legacy aircraft started with engineering drawings,
but those drawings were turned into gage tools that superseded the drawings. Once
parts were produced they could always be compared back to the gage tools if there was
a problem with the way that parts fit to the aircraft. Legacy aircraft also used drill
fixtures and routing jigs to shape parts and drill holes. With the digital thread, the hard
tooling is gone.
11
Copyright 2011 Lockheed Martin
While the benefits of the digital thread outweigh the challenges associated with it,
when there is a problem with components fitting together during the assembly process it
now becomes very difficult to tell which part is out of tolerance. This leads to the need
to inspect large assemblies such as landing gear doors, hinge assemblies, and aircraft
door openings. Measuring components like this is not something CMMs were ever
intended to do and even if an assembly such as a landing gear door was fixtured to a
CMM bed, it would take far too long to inspect enough points to get to the root cause of
the part mismatch. This has led to the development of new non-contact inspection
methods such as coherent laser radars, structured white light, and laser scanners.
While these new systems are developing rapidly, there is limited data supporting that
the accuracy of these systems is comparable to traditional CMMs. The advantage to
these non-contact systems is they are portable, much less expensive, record millions of
data points in a matter of seconds, and do not require labor intensive numerical control
programming.
While the coordinate measuring machine will retain its need in industry for a long
time to come, there is an urgent need to advance and validate the capabilities of non-
contact inspection methods in the manufacturing world.
Statement of the Problem
Coordinate measuring machines utilizing touch-probes are capable of recording
very accurate and precise measurements, but they are expensive, take up valuable
floor space, are sensitive to vibration, and only collect one point at a time; they are not
considered to be a rapid inspection technology. Non-contact inspection technologies
12
Copyright 2011 Lockheed Martin
exist that may provide solutions to these challenges, but there is limited data to support
that the non-contact systems are as accurate as touch-probe CMMs.
Purpose of the Study
The purpose of this study is to quantify the accuracies of non-contact metrology
systems utilizing structured light. Coordinate measuring machines are considered to be
one of the most capable metrology instruments available (Chapman, 2002), and are
traceable to the National Institute of Standards and Technology (NIST) standards
(Morey, 2010a), however, they are not without limitations. They require dedicated floor
space, computer programs to run them, special fixturing for the parts to be inspected,
are sensitive to vibrations, and only record a single data point each time they touch a
part. Although touch probes are capable of recording up to 60 points per minute
(Chapman, 2002), this is not fast enough for certain manufacturing applications.
Non-contact technology utilizing structured light technology, commonly referred
to as white light technology, uses a light source to project a two-dimensional shadow
pattern on a three-dimensional surface. Digital cameras then record the distorted image
that result from the contours and features of the three-dimensional surface. Powerful
software takes this image and calculates the distance the cameras are from the surface
and thus measures the part. Some of the newer systems now use blue light-emitting-
diodes (LEDs) to produce the two dimensional pattern. Advantages of blue light will be
discussed later.
According to Marc Demarest (Morey, 2010a), structured light technology has
been around since 1995, so this isn’t new technology. However, it has been gaining
13
Copyright 2011 Lockheed Martin
notoriety and feels like new technology to many users. As with any technology that is
new to a user, it is important to establish a baseline for the current state of the art and
then qualify the new technology to existing standards ensuring equivalency of
performance.
The deliverables of this study consist of gage linearity and bias study validating
the accuracy of the structured light measurement systems, and a gage repeatability and
reproducibility (gage R&R) study validating the robustness of the measurement process.
Assumptions of the Study
Gage R&R studies were not conducted on the contact metrology devices. The
CMM was calibrated in compliance with DIN EN ISO 10360-2:2001 on January 26,
2011and was accurate within 3 microns/meter. The Fowler height gage was within the
calibration certification date and verified with a NIST traceable standard. The two-ball-
over method used to measure the countersinks is considered an acceptable method of
calibrating countersink gages at Lockheed Martin.
Minitab gage R&R results no longer utilize 5.15 standard deviations to calculate
gage variation. Minitab 16 calculates study variation as 6 times the standard deviation
of each variation source in accordance with recommendations from the Automotive
Industry Action Group (AIAG) (Minitab Inc, 2010).
Limitations of the Study
This study was limited by several factors. The amount of time the CMM was
available to measure and baseline the test artifacts meant that test artifacts had to be
selected with measurement capacity in mind. Two artifacts that had originally been
scoped had to be eliminated from testing because the large CMM required to measure
14
Copyright 2011 Lockheed Martin
them was not available. CMM programmer time also affected the selection of artifacts.
Due to the lack of CMM capacity, a gage R&R was not performed for any of the contact
measurement processes. The artifact measured on the CMM was only measured once,
as was the case for the spot face panel and the countersink panel. Budget constraints
had an effect on the number of white light scanners tested and a reduction in manpower
lead to down-scoping the original project.
The time it took to analyze the data was also a limiting factor in this study. While
collecting data can be accomplished relatively quickly, analyzing point cloud data can
take a considerable amount of time. Once the process of isolating the desired features
from rest of the point cloud has been defined, an algorithm can be developed to
automate the process. However, defining the initial process can be time consuming and
may require multiple algorithms for features such as varying hole diameters.
Definition of Terms
Benchmark. “A standard by which something can be measured or judged”
(“Benchmark,” n.d., para. 1).
Bias. The difference between the measured point and the reference point
(Minitab, 2010)
Bridge-type CMM. “A type of CMM with a horizontal beam holding the probe.
The bridge-type CMM is the most common type” (“Basics,” 2011, para. 3).
Calibrated. When an instrument is compared to a more accurate instrument to
indentify and correct measurement errors (Goldsmith, 2010).
15
Copyright 2011 Lockheed Martin
Coordinate measuring machine. “A sophisticated measuring instrument with a
flat polished table and a suspended probe that measures parts in three-dimensional
space” (“Basics,” 2011, para. 3).
Digital thread. “Digital thread implies that 3D exact solid models from
engineering design are used directly by manufacturing for NC programming, coordinate
measurement machines inspections, and tooling (which are also 3D solid models)”
(Kinard, 2010, para 3).
Gage repeatability and reproducibility. Also know as gage R&R, it is a study
to “determine the magnitude of the variation in a measurement system as well as the
sources of this variation” (Kappele and Raffaldi, 2005, para 2).
Linearity. Linearity is an expression of the accuracy of the gage throughout the
expected range of measurements (Minitab, 2010)
Metrology. “The science that deals with measurement” (“Metrology,” n.d., para.
1).
Micron. One millionth of one meter. (“Micron,” n.d., para. 1).
Structured light. A pattern of light projected from either a white light source or a
blue light source onto a three dimensional surface (Morey, 2010).
Traceable. The ability to connect a measurement to a national or international
standard through an unbroken set of comparisons (Goldsmith, 2010).
Methodology
The researcher conducted a gage linearity and bias study for three structured
white light systems and compared them to contact metrology methods including a Zeiss
16
Copyright 2011 Lockheed Martin
Accura II AKTIV CMM, a Fowler digital indicator (Model 54-520-777) attached to a
height gage, and a two-ball-over method of measurement. Three test articles were
benchmarked with the contact measurement methods and then measured with a
CogniTens WLS400M Blue Light Scanner, a GOM ATOS Triple Scan with Blue Light
Technology, and a Rexcan 4 white light scanner by Solutionix. In addition to performing
the gage linearity and bias study to validate accuracy, a gage repeatability and
reproducibility (gage R&R) study was conducted. The gage R&R utilized three different
operators to measure the three test articles, each containing twenty-one measurement
points/features. Each operator measured each article three times. The results of the
gage R&R were evaluated with Minitab 16.0.0 software using the ANOVA method.
17
Copyright 2011 Lockheed Martin
Chapter II: Literature Review
As manufacturing technologies mature, and design engineers create larger and
more complex structures, the need for improved inspection technologies becomes
increasingly important. Traditional inspection equipment and processes are capable of
performing very accurate and very precise measurements, but these processes are
often relatively slow and very expensive. In addition, these processes only inspect
discrete points that may not fully reflect the condition of the part. When analyzing why
parts do not fit together properly in an assembly it is helpful to have surface data rather
than single point inspection data. A cloud of surface points may reveal a wavy, uneven
surface while the individual points gathered from a traditional CMM may be coincidently
spaced such that they reveal a near perfectly flat surface.
Three-dimensional non-contact inspection technologies have been in use for
over fifteen years but they are just now becoming capable of meeting the high accuracy
and high precision requirements of modern manufacturing industries such as
automotive and aerospace (Morey, 2010a). As a result of improving capability there is a
rush to refine and utilize these technologies to reduce manufacturing costs as well
increase manufacturing throughput. This chapter will review the current body of
literature and discuss early metrology tools, coordinate measuring machines, non-
contact metrology systems used in industry, gage repeatability and reproducibility, and
finally comments on the future of non-contact metrology.
Early Metrology Tools
The need for standard measurement devices has existed for nearly all of history
(Bucher, 2004). Methods of measuring distance were often a function of a body part
18
Copyright 2011 Lockheed Martin
such as the length of an arm or the width of a hand. Time was measured by cycles of
the sun, the moon, and the appearance of specific stars. Standard methods of
measuring food and liquids were required for cooking and bartering.
According to Bucher (2004), the use of calibrated and traceable measurement
standards goes as far back as the Egyptians. The royal cubit was made from granite
and was used as the official measurement standard. From this standard, additional
cubits were made of wood and then used in construction throughout the land. To
ensure the accuracy of the wooden cubits used by the workers, at each full moon the
cubits were returned to the royal architect and compared to the royal cubit. The
punishment for failing to return a wooden cubit was death. Because the Egyptians had
this strict system of standards they were able to build the Great Pyramid of Giza to an
accuracy of 4 ½ inches over 756 feet.
Brown and Sharpe introduced the first micrometer to the public in 1867 (Roe,
1916). While they had been in use by machinists before this time, this marked the first
time it was produced in mass. A micrometer is a handheld device consisting of a frame,
anvil, sleeve, spindle, thimble and a ratchet (Figure 1). Modern micrometers are
considered to be very precise and are accurate to ± .00005 in. (Starrett, 2011a).
Figure 1. Micrometer. Retrieved from http://www.technologystudent.com/equip1/microm1.htm
19
Copyright 2011 Lockheed Martin
While not as precise or accurate as a micrometer, another extremely useful hand
tool is the vernier caliper (Figure 2). Calipers can measure external features, internal
features, and depths to an accuracy of ±.0001 in. (Starrett, 2011b).
Figure 2. Vernier caliper. Retrieved from http://www.technologystudent.com/equip1/vernier3.htm
There are many useful hand tools available to machinists to make extremely
accurate and precise measurements, but for many of the manufacturing processes
employed today, hand tools are not adequate (Logee, Fabiano, & Cassola, 2009).
Whether it is the potential of human error, the number of points being inspected, or
shape of the part, many times automation in metrology is required.
Coordinate Measuring Machines
The coordinate measuring machine (CMM) has long been considered the
standard in metrology (O’Rourke, 2011). CMMs have been in existence since the
1960’s but have been continually improved and enhanced. According to Mark Bliek of
Bolton Works, “CMMs are still the workhorses for geometric quality control. The
calibration of a CMM can be traced back to NIST standards, and therefore the industry
is comfortable with the measurements made with a CMM” (Morey, 2010a, p.57).
Bucher (2004) says that using traceable standards for calibration is essential to
ensuring consistent parts. Using a traceable standard means that any measurement
standard used in the calibration of a measurement device can be verified by a more
20
Copyright 2011 Lockheed Martin
accurate measurement standard which can be compared to a master standard held by
an organization such as NIST. There has to be an unbroken chain of verification from
the master NIST standard down to the final equipment being calibrated. This does not
mean a machinist has to send their micrometer to NIST for calibration, but the
measurement standard they use must have been inspected by a more accurate
measurement standard that is traceable to a NIST standard. The use of a single master
measurement standard ensures that a .5 in. bolt in the United States is the same as a .5
in. bolt in Germany. This is ensured by the Bureau International des Poids et Mesures
(BIPM) in Sevres, France providing all countries with the same defined standards.
Originally CMMs were two-axis, manually driven machines. However, modern
CMMs are now computer controlled and have up to five axes or degrees of freedom
(Morey, 2010b). To ensure the CMM is performing as expected, there are several
standards that may be used to calibrate the CMM. ISO 10360-2, ASME B89.4.10360.2-
2008, and VDI/VDE 2617 are the most common standards used today (Card, 2003).
These standards are similar to each other, but they each specify a different number of
tests and different procedures for performing these tests. Over the years these
documents have adopted similar procedures to truly become standardized in areas
such as environmental controls. ISO 10360-2 used to require testing at specific
temperatures, but now just as in ASME B89.4.10360.2-2008 the user is free to choose
the environmental conditions. Nevertheless, in spite of the variations in these
documents, all standards require traceability to NIST artifacts or standards making the
CMM a highly regarded tool in metrology.
21
Copyright 2011 Lockheed Martin
CMMs come in a wide range of sizes, shapes and configurations (O’Rourke,
2011). There are small bench top CMMs that are still hand driven as well as ones that
have linear drive motors and digital readouts to make it easier to record measurements.
There are portable CMMs such as arms. The Faro Edge ScanArm weighs 24 pounds
and can be mounted to a bench top or movable stand. Mraz (2011) says the Faro arm
comes in three models that have working envelope ranges of 6 ft, 9 ft, and 12 ft. The 6
ft model has a stated single-point repeatability of .0009 in. while the 12 ft model is
repeatable to .0025 in. This means an operator measuring the same point two times
with the 6 ft model can find the same point to an accuracy of .0009 in. In addition to the
bench top and portable models, companies like Lockheed Martin have CMMs with
machine beds that that are 16 meters long, 5 meters wide, and capable of measuring
parts 2.5 meters tall with a linear accuracy of 10 microns per meter (B. Kush, personal
communication, August 1, 2011).
Coordinate measuring machines are very accurate and useful for measuring
parts, but each part measured requires its own computer program to run the machine.
Some parts may only take a couple of hours to program but Mark Boucher (2007) of
NewCastle Measurement says developing CMM a program for an extremely complex
part could take several days to complete. In addition to the lengthy programming time
the CMM also may require numerous calibration routines to be developed to account for
and reduce the risk of probe offset errors. Programming can often be performed off line
which saves CMM time, but still requires the expense of a CMM programmer. Software
is constantly being improved to make programming easier, but often the CMM programs
are not compatible with CMMs from different manufacturers. This means if a shop has
22
Copyright 2011 Lockheed Martin
multiple CMMs from different manufacturers, the CMMs must have dedicated parts or
multiple programs will be required. CMMs are very useful, but not always flexible and
accommodating to change.
Non-Contact Metrology
As the automotive, aerospace, and wind turbine industries move forward with
more advanced designs and manufacturing processes, it becomes clear that even
though the CMM will not be disappearing anytime soon, new methods of inspection will
be required. The aerospace industry and the wind turbine industry both use very large
monolithic composite structures. According to Zach Rodgers at Nikon Metrology,
whether it is a barrel section of a commercial plane’s fuselage, or a long wind turbine
blade, measuring large structures like these is just not feasible with a CMM (Morey,
2011, p. 59-60). In both automotive and aerospace there is a need to rapidly inspect
complex shapes that would be impractical to fixture on a CMM (Morey, 2010b). To
measure parts in this complex environment, the need for non-contact inspection
technologies is rapidly growing.
In 2004, Lockheed Martin Aeronautics determined they had to develop an
alternative approach to controlling the thickness of composite wing skins (P. Briney,
personal communication, July 7, 2011). Lockheed’s previous approach utilized very
expensive milling machines to profile the inner mold line of the laminate which left
structurally unnecessary material on the part resulting in excessive weight. To
overcome this challenge, the cured laminate compensation (CLC) process was
developed. The CLC process was made possible through the use of the Nikon MV-224
Coherent Laser Radar (CLR).
23
Copyright 2011 Lockheed Martin
The MV-224 is a form of non-contact CMM and has been replaced by the next
generation Nikon MV-330. Originally developed by Coleman Research in the early
1990’s, the laser radar has been in continual use and undergoing improvements ever
since (P. Morken, personal communication, July 7, 2011). According to the Nikon Laser
Radar Training Workbook (2010) the MV330 operates on a principle similar to the
conventional radar principle of time-of-flight. To do this, a laser beam is split into two
beams with one beam traveling through a reference coil of known length and the other
beam traveling to the surface. Distance is calculated by measuring the time it takes for
the laser signal to reflect back from the surface being measured, and comparing this
signal to the time it takes the second signal to travel though the reference coil. Since
time-of-flight can be difficult to measure, the laser signal is modulated to vary its
frequency. The change in the frequency has a direct relationship to time-of-flight and is
much easier to measure. It is through the development of proprietary software that
Nikon has been able to achieve this. The MV-330 has a stated accuracy of .001 in. at a
distance of 6 ft, and .004 in. at 100 ft. It was this level of accuracy and the
programmable non-contact features that enabled Lockheed Martin to develop a solution
to their composite thickness challenge. The Laser Radar can be operated in a manual
mode or programmed just like the touch-probe CMMs, enabling them to consistently
measure the same discrete points. Jeff Drewett of Lockheed Martin says the CLR
makes the CLC process repeatable to .0013 in. on composite surfaces and .0008 in. on
an Invar cure tool surface (Morey, 2011, p. 63).
The latest technology to make headlines in metrology is structured light.
Structured light is not a new technology, but it has gained notoriety in the last couple of
24
Copyright 2011 Lockheed Martin
years (Morey, 2011). It is commonly referred to as white light scanning even with the
recent introduction of blue light scanners. Hexagon Metrology has unveiled a new
scanner utilizing blue light-emitting-diodes (LEDs) as the light source, but they branded
the product as the WLS400 to take advantage of the white light name.
The terms white light system and structured light system have become a sort of
catch-all for several technologies (3D Surface Reconstruction Technologies, 2008).
Currently there are systems utilizing white light and systems using blue light. White light
was the first to be developed, but white light systems can be affected by shiny surfaces.
Blue light scanners filter out all wave-lengths except blue light, and are therefore less
sensitive to ambient light and highly reflective surfaces. The CogniTens WLS400M blue
light scanner has been shown to produce less measurement variation than the
CogniTens Optigo 200 which uses white light (C. Bliss, personal communication, July
26, 2011).
In addition to the difference in the light source, there is also a difference in the
way the data is collected. While structured light is often used to describe these
systems, some systems use structured light and others use stereo vision (3D Surface
Reconstruction Technologies, 2008). Structured light projects a series of moving gray
stripes on a three dimensional surface. As the pattern distorts over the three
dimensional surface, high resolution digital cameras record the data (Morey, 2011).
With the aid of powerful software, the shape of the object is determined from the
distorted gray stripes. Structured white light systems can utilize one, two, or three
imaging cameras and a light source. Structured light systems can produce very good
data with little variation, but the time it takes to acquire the data can be two or three
25
Copyright 2011 Lockheed Martin
seconds. Because of this, structured light systems must be very stable as they are
susceptible to vibrations. If the part moves or the camera system moves while the data
is being acquired, the recorded image will be inaccurate. Stereo vision systems use two
or three digital imagining cameras to record the data, but rather than project a moving
two-dimensional pattern of gray bars over the three-dimensional surface, stereo vision
systems project a static pattern (3D Surface Reconstruction Technologies, 2008). If the
two cameras are able to see a common point in two stereo images, they can use
triangulation to calculate the location of the point in space. Because the projected
image does not move and the camera acquires the image in milliseconds, this type of
white light system is much less sensitive to vibration. However, the trade off is the data
acquired has more variation than the structured light system. Because of this, the
application must be considered. While stereo vision may not provide as precise data as
structured light, it works very well in a factory floor environment.
The manufactures of structured light systems claim the accuracy is equivalent to
touch-probe CMMs, but with the exception of the work by Hammett and Garcia-Guzman
(2006), few significant studies have been carried out to date. Also, as Hammett and
Garcia-Guzman (2006) conclude, there is more to determining the accuracy of a system
than just measuring an artifact. The entire process of acquiring the data must be
considered. According to a study by Hammett, Guzman, Frescoln, and Ellison (2005),
one must consider how the data is collected. One advantage of non-contact systems
over traditional CMMs is the part can often be measured without tooling or fixtures to
hold the part. In these examples, the data is analyzed relative to the part model and
can be extremely accurate. However, if the part is held in a fixture and analyzed
26
Copyright 2011 Lockheed Martin
relative to a fixtured coordinate system, the capability of the system is reduced.
Hammett theorizes that 80-90% of the part variation will come from the set-up in the
fixture. Because of this, great care must be taken to ensure the right process and
measurement systems are used for each specific application.
Gage Repeatability and Reproducibility
To validate a measurement process, a gage repeatability and reproducibility
(gage R&R) study should always be performed (Sloop, 2009). A gage R&R study is
designed to identify how much variation is in a measurement process and where the
variation comes from. Measurement variation comes from three main sources and it is
important to identify how much variation comes from each source. By knowing the
sources of variation, it may be possible to improve the measurement process (Kappele
& Raffaldi, 2005).
While there is not one single standard for performing a gage R&R, a typical study
consists of three operators, measuring ten features or parts, three times (Morey,
2010c). Repeatability is measured by the consistency of each operator’s
measurements. Reproducibility is measured by the degree to which the different
operator’s measurements agree with each other. High reproducibility with low
repeatability could indicate the operators need more training, while high repeatability
compared to low reproducibility could indicate poor equipment (Kappele & Raffaldi,
2005). Performing a gage R&R is a critical step to building confidence in the
measurement process.
27
Copyright 2011 Lockheed Martin
The Future of Non-Contact Inspection
Non-contact metrology systems have made tremendous advances in the last ten
years and are taking on an increasing role in manufacturing (Morey, 2009). In-process
inspection is now being integrated as another part of the manufacturing process rather
than something performed only after the part is fabricated. By involving metrology
earlier in the manufacturing process, problems can be identified and corrected quickly to
minimize the cost of scrap and rework. It seems clear non-contact inspection systems
will continue to take on an expanding role in manufacturing. As more companies in the
manufacturing industry utilize this technology, it will be more critical than ever to have
standards and common processes for validating the capability of the equipment to
ensure it is matched with the proper application and need. There have been few
studies evaluating the performance of non-contact metrology relative to proven contact
metrology systems. However, with the increased interest and awareness of these
systems, it seems clear more and more data will be presented in the future.
28
Copyright 2011 Lockheed Martin
Chapter III: Methodology
The purpose of this study was to evaluate the accuracy and repeatability of non-
contact metrology systems utilizing white light systems by comparing those results to
the accuracy and repeatability of coordinate measuring machines utilizing touch probes.
Coordinate measuring machines have a long history of reliable use and are traceable to
NIST standards, but they are expensive, relatively slow, and require dedicated floor
space. In some instances, they require environmentally controlled rooms. As various
industries produce larger monolithic structures such as wind turbine blades, airplane
fuselages, and wings, the need to control both cost and part quality with advanced,
rapid-inspection technologies becomes greater.
This chapter explains the process used to benchmark the capability of the
coordinate measuring machine, as well as how the data was collected with the non-
contact systems. In addition, it addresses the equipment used, how the test articles
were selected, and the number of operators collecting data. Finally, the method of the
data analysis for the gage R&R study will be discussed.
Equipment
There are many types of CMMs available and many types of white light systems
to choose from. For the baseline CMM test a Zeiss Accura II AKTIV CMM was used
(Figure 3). The Zeiss has an inspection bed that is 2 meters wide, 3 meters long and
can measure parts 1.5 meters tall. It has a stated linear accuracy of 10 microns per
meter, but calibration records indicate it is accurate to 1.5 microns per meter.
29
Copyright 2011 Lockheed Martin
Figure 3: Zeiss Accura II AKTIV CMM
To evaluate non-contact white light systems, three different systems were
selected.
CogniTens WLS 400M White Light Scanner
ATOS Triple Scan with Blue Light Technology
Rexcan 4 White Light Scanner
The first system evaluated was a WLS400M white light system manufactured by
CogniTens, a subsidiary of Hexagon Metrology (Figure 3).
30
Copyright 2011 Lockheed Martin
Figure 4. CogniTens WLS 400M
The WLS 400 uses blue light emitting diodes (LEDs) as a light source but is still
considered to be a white light system. Utilizing three cameras and stereo vision as the
method of data collection the WLS 400M projects a static pattern which is captured in
milliseconds and is therefore impervious to vibrations.
The second system used in the study was the ATOS Triple Scan with Blue Light
Technology by Gesellschaft für Optische Messtechnik (GOM) (Figure 4). The ATOS
Triple Scan was selected because it uses structured light and claims to have higher
accuracy and a lower signal-to-noise ratio, meaning the data should contain fewer
outliers than stereo vision systems. The ATOS uses blue LEDs as the light source and
has two cameras.
31
Copyright 2011 Lockheed Martin
Figure 5. ATOS Triple Scan
The third white light system selected was the Rexcan 4 made by Solutionix. The
Rexcan 4 white light scanner is much less expensive than any of the other systems so it
was selected to evaluate what its capabilities are. This system uses two cameras and a
metal halide light source to project a white light pattern on the part and measure the
phase shift of the pattern.
32
Copyright 2011 Lockheed Martin
Figure 6. Rexcan 4 White Light Scanner
Features and Test Articles
This study arose from the need to inspect specific features on large components
in a rapid manner with minimal to no fixturing. A list of these features was compiled and
used to drive the selection of the test articles.
The main features of concern were:
Surface profile
Fastener depth
33
Copyright 2011 Lockheed Martin
Countersink diameter
To reduce the time and cost of designing and fabricating test articles, existing tools and
artifacts were used whenever possible.
Three test articles used were:
Precision Surface Block
Spot Face Panel
Countersink Panel
The first test article used was the precision surface block (Figure 6). This tool
had been fabricated several years earlier and provided a very stable tool to validate the
ability of the various systems to measure a surface profile. The 9 in. by 5 in. aluminum
block had three precision tooling balls bonded into bushings and a gentle contour
machined into the upper surface.
Figure 7. Precision Surface Block
The second tool selected was a simple gage tool and was referred to as the spot
face panel (Figure 7). It was fabricated from a Blanchard ground steel plate with a
34
Copyright 2011 Lockheed Martin
series of spot faces machined into the surface to represent fastener heads. The
purpose of the gage tool was to provide a simple tool with as little variation as possible
to assess accuracy and repeatability. Two thirds of the tool was coated with black oxide
and one third of the tool was painted with green aircraft primer to evaluate the effects of
shininess. A drawing of the gage tool can be found in Appendix A.
Figure 8. Spot Face Panel
The last artifact, known as the countersink panel, was created from a 12 in.
square graphite-epoxy test panel. The test panel was .500 in. thick and had twenty-one
countersinks drilled into the surface and twelve non-countersunk holes. The
countersinks ranged from .350 in. to .750 in. in diameter. The panel was painted with
green aircraft primer, and the countersinks were machined after the primer was applied
so the resulting countersinks were black. This tool was originally fabricated to ascertain
the ability of white light systems to determine the depth and diameter of countersinks,
35
Copyright 2011 Lockheed Martin
the diameter of drilled holes, and locations of both holes and countersinks, however,
due to time limitations only countersink diameters were evaluated. A drawing of the
panel is included in Appendix B.
Figure 9. Countersink Panel
Benchmarking
Each test article was built in accordance to a design model, but since the study
was not concerned with the capability of the machines producing the test articles, the
only values of concern were the as-built nominal values. Once the features were
measured by the contact measurement methods these were the values by which all
other analysis would be compared to. In order to evaluate the performance of the non-
contact systems it was critical to have reliable nominal values of the features and test
articles.
The first test artifact was the precision surface block. A computer model was
provided to the CMM programmer and an inspection routine was developed. Once the
CMM program was transferred to the CMM controller, the test article was inspected.
36
Copyright 2011 Lockheed Martin
.
The surface and spot faces of the spot face panel were inspected with a Fowler
digital indicator (Model 54-520-777) attached to a height gage. The indicator was
zeroed on the top of the surface plate and then measurements were recorded in four
locations. These four measurement values were averaged to become the nominal spot
face depth.
To baseline the countersink panel a two-ball, ball over method was used to
measure the countersink diameter of the countersink at the surface. Using this method,
two spheres were placed one at a time in the countersink. With the sphere in the
countersink, the height of the sphere relative to the top of the countersink was recorded.
This process is repeated for the second sphere. Using the formula in Appendix F the
diameter of each countersink was calculated.
Non-Contact Data Collection
The collection of the non-contact measurement data took place over a course of
a two week period with one system tested the first week and two systems tested the
following week. The vendors shipped their equipment to the Lockheed facility on the
Friday before testing which meant Monday morning was spent unpacking and
calibrating the equipment. Measurements started Monday afternoon and continued
through Friday morning allowing for time to pack up the equipment and ready it for
shipment.
Measurement data from white light systems is captured by cameras and
triangulated by software, and therefore limited by the field of view of the camera at a
given focal length. To overcome this limitation, some systems utilized photogrammetry
37
Copyright 2011 Lockheed Martin
to map the artifacts and provide a scaled reference frame. Each vendor utilized their
own photogrammetry equipment.
The first system tested was the CogniTens WLS 400M which was calibrated per
the manufacturer’s specifications prior to the start of the measurements.
Photogrammetry targets were placed on the parts and the table. The set-up was then
mapped to ensure multiple shots could be stitched together if required.
Figure 10. Calibrating the WLS 400M
The Precision Surface Block was placed on a sturdy steel table and then
measured by three operators. Each operator created three measurement collections
and each collection consisted of two shots from slightly different positions. By taking
two shots per collection the measurement point data was doubled to help ensure
sufficient data was collected. The WLS 400M is a portable system so prior to each
shot, the operator moved to camera to introduce some degree of variability into the
38
Copyright 2011 Lockheed Martin
process. The actual acquisition of the measurement is taken by pointing the camera at
the part and pressing a button, so variation form the operators should be minimal.
Figure 11. Measuring the Precision Surface Block with the WLS 400M
The Spot Face Panel was the next artifact measured. This artifact was
measured in the same manner as the Precision Surface Block; three operators, each
taking three collections. As before, each collection consisted of two shots to ensure
adequate point density.
Figure 12. Measuring the Spot Face Panel with the WLS 400M
39
Copyright 2011 Lockheed Martin
The last artifact to be measured was the countersink panel. Countersinks pose a
big challenge to non-contact systems. Without sufficient point data, the measurement
system may not actually collect a data point at the intersection of the edge of the
surface and the start of the countersink. Accurately measuring features such as
countersinks may require additional post processing software such as PolyWorks or
Geomagic. These software programs are better suited to best-fit a cone or similar
feature into the data for comparative purposes. While countersink measurement can be
a difficult task, it is one that aerospace manufacturers would benefit greatly from.
Figure 13. Measuring the Countersink Panel with the WLS 400M
During the second week of testing two systems were evaluated. The Rexcan 4
by Solutionix and the ATOS Triple Scan by GOM were both evaluated using the same
40
Copyright 2011 Lockheed Martin
procedures as the previous two systems. For these two systems photogrammetric
mapping was used to stitch multiple images together. Once the parts were mapped, the
data acquisition followed with the Precision Surface Block first, the Spot Face Panel
second and the Countersink Panel last.
Figure 14. Calibrating the Rexcan 4 White Light Scanner
Figure 15. Mapping the Artifact with Photogrammetry
41
Copyright 2011 Lockheed Martin
Figure 16. Measuring the Precision Surface Block with the Rexcan 4 White Light
Scanner
Figure 17. Measuring the Spot Face Panel with the Rexcan 4 White Light Scanner
42
Copyright 2011 Lockheed Martin
Figure 18. Measuring the Countersink Panel with the Rexcan 4 White Light Scanner
Figure 19. Measuring the Precision Surface Block with the ATOS Triple Scan
43
Copyright 2011 Lockheed Martin
Figure 20. Measuring the Spot Face Panel with the ATOS Triple Scan
Figure 21. Measuring the Countersink Panel with the ATOS Triple Scan
44
Copyright 2011 Lockheed Martin
Gage Repeatability and Reproducibility Study
A gage repeatability and reproducibility (gage R&R) study should always be
performed for new inspection processes. The gage R&R is used to determine if the
measurement system is capable of providing correct inspection data for the tolerances
specified. As noted in the literature review, there is no single specification to guide the
gage R&R study. For this study three operators were selected to collect the data with
the non-contact systems. Each operator measured each test article three times,
capturing twenty-one features or measurement points. A matrix of the data collection
forms can be found in the Appendix of this report. Once the data was collected, Minitab
16.2.0.0 software was used to analyze the data using the ANOVA method. The output
from Minitab provided information on the range of the measurement variation, the
standard deviation of the results and a precision to tolerance ratio that indicates how
capable the measurement system is for the intended use. Linearity and bias data were
also provided.
45
Copyright 2011 Lockheed Martin
Chapter IV: Results
Non-contact methods of inspection have been developed over the last fifteen
years, but there is limited data comparing the accuracy of these technologies to contact
methods of inspection. This study was conducted to make such a comparison.
Results
The analysis of this study was conducted using Minitab 16.2.0.0 software and is
contained in the appendix of this paper.
Summary of Findings
Three non-contact metrology systems were used to measure three distinct test
artifacts.
Non-contact Metrology Systems:
1. CogniTens WLS 400M White Light Scanner
2. ATOS Triple Scan With Blue Light Technology
3. Rexcan 4 White Light Scanner
Test Artifacts:
1. Precision Surface Plate
2. Spot Face Panel
3. Countersink Panel
The Precision Surface Plate evaluated the ability of each system to measure a
contoured surface and relate the points back to a nominal solid model. The Spot Face
Panel simulated measuring installed fastener depths, and the Countersink Panel tested
the ability to measure the diameter of a countersink. The Gage R&R was performed
46
Copyright 2011 Lockheed Martin
utilizing the tolerances in Table 1 which are listed in inches. The full gage study results
are provided in the appendix of this paper.
Table 1: Test Article Tolerances Test Article Lower Tolerance Upper Tolerance
Precision Surface Block -.005” .005”
Spot Face Panel .000” .020”
Countersink Panel -.010” .010”
Gage Linearity and Bias Study
Gage linearity and bias are measurements of how accurate the measurement
system is relative to the baseline or reference value and how accurate it is across the
full range of intended measurements. Bias indicates the accuracy and linearity tells
how consistent the system is. The following table presents the bias of each system
relative to the nominal values. If the bias is higher than zero the system overestimates
the measurement and if the bias is negative it underestimates the measurement
(Minitab, 2007).
Table 2: Average Bias of Each Measurement System Test Article CogniTens WLS
400M ATOS Triple
Scan Rexcan 4 White
Light System Precision Surface Block -0.00010” 0.00015” 0.00028”
Spot Face Panel 0.00009” -0.00010” 0.00087”
Countersink Panel 0.00285” -0.00032” 0.00166”
Gage linearity describes how accurate the measurement system is across the
range of measurement process. Linearity is expressed as the slope of the
measurement bias as well as a %Linearity, which is linearity expressed as a percent of
47
Copyright 2011 Lockheed Martin
the process variation (Minitab, 2007). Linearity data is provided in Table 3. Small
linearity values indicate the accuracy remains the same throughout the measurement
range.
Table 3: Linearity Values for Each System Measuring and Each Test Article Test Article CogniTens WLS
400M ATOS Triple
Scan Rexcan 4 White
Light System Precision Surface Block 0.00120 0.00175
0.00051
Spot Face Panel 0.00029 0.00056 0.00482
Countersink Panel 0.02110 0.01197
0.00140
%Linearity provides a metric to show measurement consistency. The closer %Linearity
is to zero, the more consistently the gage measures across the measurement range
(Minitab, 2007). Table 4 provides %Linearity data.
Table 4: %Linearity Values for Each System Measuring and Each Test Article Test Article CogniTens WLS
400M ATOS Triple
Scan Rexcan 4 White
Light System Precision Surface Block 2.20%
3.10%
0.90%
Spot Face Panel 0.07%
1.30%
12.10%
Countersink Panel 2.50%
1.50%
0.20%
Gage Repeatability and Reproducibility Study
A gage R&R study is useful to determine the amount of variation in a
measurement device or system as well as the source of the variation. Measurement
repeatability measures the variation between measurements taken by a single operator
and should be low. Table 5 provides the measurement repeatability for the systems.
48
Copyright 2011 Lockheed Martin
Table 5: Gage R&R Repeatability Results Test Article CogniTens WLS
400M ATOS Triple
Scan Rexcan 4 White
Light System Precision Surface Block 0.06%
0.01%
0.92%
Spot Face Panel 0.52%
0.21%
18.57%
Countersink Panel 0.02%
0.01%
0.09%
Gage Reproducibility measures the amount of variation between operators
measuring the same part. Table 6 provides reproducibility data.
Table 6: Gage R&R Reproducibility Results Test Article CogniTens WLS
400M ATOS Triple
Scan Rexcan 4 White
Light System Precision Surface Block 0.02%
0.00%
0.12%
Spot Face Panel 0.05%
0.10%
1.21%
Countersink Panel 0.00%
0.03%
0.01%
Typically, the total gage variation for a measurement system would equal less
than 30% of the tolerance while less than 10% would be preferred (Minitab, 2007).
Table 7 provides the total gage variation as a percent of the tolerance.
Table 7: Total Gage R&R Variation as A Percent of the Tolerance
Test Article CogniTens WLS 400M
ATOS Triple Scan
Rexcan 4 White Light System
Precision Surface Block 15.18%
5.94%
55.40%
Spot Face Panel 15.33%
11.45%
88.88%
Countersink Panel 64.58%
80.30%
179%
49
Copyright 2011 Lockheed Martin
Table 8 provides a breakdown of measurement system acceptability based on a 30%
variation/tolerance ratio.
Table 8: Acceptability of the Measurement Systems Based On 30% Variation/Tolerance Ratio Test Article CogniTens WLS
400M ATOS Triple
Scan Rexcan 4 White
Light System Precision Surface Block Acceptable
Acceptable
Unacceptable
Spot Face Panel Acceptable
Acceptable
Unacceptable
Countersink Panel Unacceptable
Unacceptable
Unacceptable
50
Copyright 2011 Lockheed Martin
Chapter V: Discussion
Non-contact metrology devices have made tremendous advances in the last ten
years and offer potential solutions to manufacturing challenges. Allowing quick
inspection of complex parts earlier in the manufacturing process offers the opportunity
to reduce scrap and increase part quality. Before any new inspection technology can
be utilized in an inspection process the system must be qualified for that given process.
This study evaluated the ability of three white light measurement systems to measure
surface contour, spot face depth (simulating fasters in a structure) and countersinks.
Conclusions
The results of this study show that the CogniTens WLS 400M and the GOM
ATOS Triple Scan are capable of accurately and repeatedly measuring surface profiles
and spot faces within the tested measurement tolerance range. However, neither
system was capable of measuring countersink diameters using point cloud data for
analysis. The Rexcan 4 White Light Scanner was not capable of achieving reliable
results for any of the measured test article.
Recommendations
The Rexcan 4 White Light Scanner was included in this study because it costs
significantly less than either the CogniTens or the ATOS scanners. Testing it against
the tight tolerances in this study showed it is not a capable instrument for this purpose.
However, given a wider tolerance band and different application, the Rexcan 4 may
show to be a cost effective alternative.
51
Copyright 2011 Lockheed Martin
The CogniTens WLS 400M and GOM ATOS Triple Scan both proved to be
capable systems but each system has strengths and weakness that also need to be
considered for each application. While the ATOS system has slightly better results, the
ATOS system is sensitive to vibration and cannot be hand held. Since the CogniTens
WLS 400M acquires measurement data in milliseconds it can be hand held and can be
used in a wider range of production-type environments. In addition, the CogniTens
WLS 400M has a Vision Mode feature that allows the scanner to look for contrast
details on the surface. Countersink measurement data using this feature wasn’t
included in this study since the other two systems do not have it, but using the vision
mode offered much improved measurement data on the painted countersink panel.
However, if the panel had not been painted and there was not significant color contrast
between the countersink and the surface, it is not clear the vision mode advantage
would be maintained.
Collecting measurement data is relatively easy but knowing how to analyze takes
effort. While the ATOS and CogniTens systems were demonstrated to be capable,
there is certainly a need for improved algorithms to speed up the analysis of the data.
Now that the equipment has been shown to be capable, a follow-on study should be
conducted to analyze the measurement process to achieve optimized results.
52
Copyright 2011 Lockheed Martin
References
3D surface reconstruction technologies. (2008). North Kingstown, R.I.: Hexagon Metrology.
Basics of the cmm 120: Inspection training. (2011). Retrieved from
http://www.toolingu.com/definition-350120-22452-coordinate-measuring-machine.html.
Benchmark. (n.d.). In The free dictionary by farlex. Retrieved from
http://www.thefreedictionary.com/benchmark.
Boucher, M. (2007, July). Using offline programming keeps that cmm on track. Tooling and
Production. 73. (7). 32-33.
Bucher, J.L. (Ed.). (2004). The metrology handbook. Milwaukee, WI, ASQ Quality Press.
Card, G. (2003, June) Selecting your cmm. Manufacturing Engineering, 130, (6) 71-76, 78.
Chapman, W. (Ed.). (2002). Modern machine shop: Handbook for the metalworking industries.
Cincinnati, OH. Hanser Gardner Publications.
Goldsmith, M. (2010). Good practice guide no. 118: A beginner’s guide to measurement. United
Kingdom, Queen’s Printer and Controller of HMSO.
Hammett, P.C. & Garcia Guzman, L. (2006). White light measurement: A catalyst for change in
automotive body dimensional validation: Measurement strategies for stamping and body
assembly from tryout through PPAP. (UMTRI-2006-3). Ann Arbor, Michigan.
Hammett, P.C., Guzman, LG., Frescoln K.D. & Ellison, S.J. (2005). Proceedings from SAE
World Congress. Changing automotive body measurement system paradigms with 3D
non-contact measuring systems. Warrendale, PA: SAE International.
Kalpakjian, S. (1992). Manufacturing engineering and technology. (2nd ed.). Reading,
Massachusetts, Addison-Wesley Publishing Company.
53
Copyright 2011 Lockheed Martin
Kappele, W.D. & Raffaldi, J.D. (2005, December). An introduction to gage r&r. Quality, 44
(13), 24-25.
Kinard, D. (2010, September). The digital thread - Key to F-35 joint strike fighter affordability.
Aerospace Manufacturing and Design. Retrieved from
http://www.onlineamd.com/Author.aspx?AuthorID=5187
Logee, S., Fabiano, R., & Cassola, J. (2009). Little CMM that could. Tool and Production, 75,
(5/6), 32-33.
Metrology. (n.d.). In The free dictionary by farlex. Retrieved from
http://www.thefreedictionary.com/metrology.
Micron. (n.d.). In The free dictionary by farlex. Retrieved from
http://www.thefreedictionary.com/micron.
Minitab Inc. (2007). Minitab Statistical Software, Release 15 for Windows, State College,
Author, PA.
Minitab Inc. (2010). Gage studies for continuous data, Release 16, Version 1.0, State College,
Author, PA. Retrieved from
http://www.minitab.com/uploadedFiles/Shared_Resources/Documents/Sample_Materials
/TrainingSampleMeasurementSystemsMTB16EN.pdf
Morey, B. (2009, November). Shop floor metrology. Manufacturing Engineering, 143, (5), 55-
60.
Morey, B. (2010a, July). Metrology with white light. Manufacturing Engineering, 145, (1), 55-
61.
Morey, B. (2010b, September). Advances in automotive metrology. Manufacturing Engineering,
145, (3), 53-61.
54
Copyright 2011 Lockheed Martin
Morey, B. (2010c, November). Accuracy and uncertainty in non-contact metrology.
Manufacturing Engineering, 145, (5), 67-74.
Morey, B. (2011, March). Aerospace metrology tools for the working day. Manufacturing
Engineering, 143, (3), 53-65.
Mraz, S.J. (2011, July). Cmm gets easier to use. Machine Design, 83, (12), 18.
Nikon Laser Radar Training Workbook (2010). Brighton, MI.: Nikon Metrology.
O’Rourke. K. (2011, April). Strength in numbers. Quality, 50, (4), 36-38.
Roe, J.W. (1916). English and American tool builders, Yale University Press.
Sloop, R. (2009, September). Understand gage r&r. Quality, 48, (9) 44-47.
Starrett (2011a). General micrometer information. Retrieved from
http://www.starrett.com/download/222_p1_5.pdf
Starrett (2011b). Slide Calipers. Retrieved from
http://www.starrett.com/download/246_p108_114.pdf
56
Copyright 2011 Lockheed Martin
Appendix B: Spot Face Panel Benchmark Results
Dia. (Column) A - Depth B - Depth C - Depth D - Depth Averaged Nominal 0.375” (1) 0.0006” 0.0006” 0.0007” 0.0006” 0.0006 0.375” (2) 0.0011” 0.0012” 0.0011” 0.0011” 0.0011 0.375” (3) 0.0038” 0.0039” 0.0037” 0.0034” 0.0037 0.375” (4) 0.0075” 0.0079” 0.0080” 0.0073” 0.0077 0.375” (5) 0.0113” 0.0118” 0.0118” 0.0112” 0.0115 0.375” (6) 0.0152” 0.0154” 0.0156” 0.0156” 0.0155 0.375” (7) 0.0192” 0.0193” 0.0197” 0.0195” 0.0194
0.500” (1) 0.0009” 0.0009” 0.0010” 0.0007” 0.0009 0.500” (2) 0.0014” 0.0016” 0.0018” 0.0015” 0.0016 0.500” (3) 0.0032” 0.0038” 0.0039” 0.0032” 0.0035 0.500” (4) 0.0073” 0.0080” 0.0080” 0.0070” 0.0076 0.500” (5) 0.0105” 0.0107” 0.0109” 0.0108” 0.0107 0.500” (6) 0.0156” 0.0154” 0.0156” 0.0159” 0.0156 0.500” (7) 0.0193” 0.0193” 0.0196” 0.0194” 0.0194
0.625” (1) 0.0013” 0.0016” 0.0015” 0.0012” 0.0014 0.625” (2) 0.0018” 0.0019” 0.0022” 0.0018” 0.0019 0.625” (3) 0.0031” 0.0043” 0.0032” 0.0031” 0.0034 0.625” (4) 0.0077” 0.0078” 0.0073” 0.0072” 0.0075 0.625” (5) 0.0122” 0.0120” 0.0115” 0.0115” 0.0118 0.625” (6) 0.0161” 0.0152” 0.0154” 0.0164” 0.0158 0.625” (7) 0.0195” 0.0184” 0.0187” 0.0194” 0.0190
59
Copyright 2011 Lockheed Martin
Appendix E: Two-Ball, Ball Over Method Formula
0 1 . d2
p,w: (H)-01/2 H I\2 ·d2/2l
1 0112 e: : 2 )( \ sin<J.: I) tanw • IH)-0112 X
60
Copyright 2011 Lockheed Martin
Appendix F: Input Data and Results for Precision Surface Panel
CogniTens WLS 400M Input Data
Operator 1 Operator 2 Operator 3
Nominal Rep 1 Rep 2 Rep 3 Rep 1 Rep 2 Rep 3 Rep 1 Rep 2 Rep 3
0.1024 0.10419 0.10412 0.10415 0.10395 0.10389 0.10386 0.1041 0.10385 0.10371
0.0952 0.09654 0.09649 0.09658 0.09666 0.09617 0.09645 0.09658 0.09637 0.09621
0.0933 0.09437 0.09436 0.0944 0.09433 0.09403 0.09414 0.09458 0.09429 0.0942
0.0930 0.0938 0.09385 0.09395 0.09369 0.09366 0.09352 0.0939 0.09383 0.09421
0.0928 0.09342 0.09313 0.09344 0.09339 0.09305 0.09317 0.09335 0.09311 0.09421
0.0929 0.09285 0.09279 0.09299 0.09295 0.09246 0.09267 0.09296 0.09276 0.09371
0.0950 0.09444 0.09432 0.09453 0.09451 0.09407 0.09414 0.09466 0.09435 0.09491
0.0946 0.09544 0.09543 0.0955 0.09551 0.09532 0.09535 0.09555 0.09531 0.09512
0.0997 0.09995 0.10006 0.10005 0.10002 0.09977 0.0999 0.10012 0.09985 0.09974
0.1027 0.1027 0.10281 0.10282 0.10268 0.10249 0.10255 0.10311 0.10277 0.10297
0.1020 0.10195 0.10182 0.10202 0.10194 0.10168 0.10156 0.10218 0.10188 0.10283
0.0976 0.09691 0.09671 0.09698 0.0971 0.09658 0.09676 0.09706 0.09676 0.09775
0.0897 0.08896 0.0889 0.08908 0.08916 0.08865 0.0888 0.08927 0.08901 0.08927
0.0821 0.08098 0.08091 0.08117 0.0812 0.08068 0.08073 0.08132 0.08094 0.08106
0.1012 0.10141 0.10145 0.10149 0.10149 0.10133 0.10129 0.10158 0.10127 0.10113
0.1093 0.10884 0.1089 0.10891 0.109 0.10873 0.10891 0.10921 0.10885 0.1088
0.1120 0.11111 0.11121 0.11127 0.11126 0.11104 0.11104 0.11168 0.11127 0.11176
0.1080 0.10679 0.10691 0.10701 0.10708 0.10684 0.10665 0.10726 0.10692 0.10801
0.0977 0.09649 0.09653 0.09674 0.09687 0.09635 0.09653 0.0969 0.0966 0.097
0.0848 0.08329 0.08339 0.08352 0.08358 0.08324 0.08317 0.08384 0.08351 0.08347
0.0735 0.07225 0.07209 0.07243 0.07245 0.07195 0.0719 0.0727 0.07227 0.07232
61
Copyright 2011 Lockheed Martin
0.110.100.090.080.07
0.002
0.001
0.000
-0.001
-0.002
Reference Value
Bia
s
0
Regression
95% CI
Data
Avg Bias
BiasLinearity
2
1
0
Pe
rce
nt
C onstant -0.0022090 0.0006800 0.001
Slope 0.021889 0.007041 0.002
Predictor C oef SE C oef P
Gage Linearity
S 0.0008577 R-Sq 4.9%
Linearity 0.0012239 %Linearity 2.2
A v erage -0.0001040 0.2 0.000
0.0735 -0.0012378 2.2 0.000
0.0821 -0.0011011 2.0 0.000
0.0848 -0.0013544 2.4 0.000
0.0897 -0.0006889 1.2 0.000
0.0928 0.0005633 1.0 0.003
0.0929 0.0000044 0.0 0.975
0.093 0.0008233 1.5 0.000
Reference Bias %Bias P
Gage Bias
Gage name: C ogniTens WLS 400M
Date of study : Nov 2011
Reported by : M. Hestness
Tolerance: ±.005
Misc: Precision Surface Block
Percent of Process Variation
Gage Linearity and Bias Study for Measured
Part-to-PartReprodRepeatGage R&R
400
200
0
Perc
ent
% Contribution
% Study Var
% Tolerance
Point
9
Point
8
Point
7
Point
6
Point
5
Point
4
Poin t
3
Point 21
Point 20
Point
2
Point 19
Point 1 8
Point 17
Point 16
Point 15
P oin t
14
Point 13
Point 12
Point 11
Point 10
Point
1
Point
9
Point
8
P oint
7
Point
6
Point
5
Point
4
Point
3
P oin t
21
Point 20
Point
2
Point 19
Point 18
Point 1 7
Point 16
Point 15
Point 14
Point 1 3
Point 12
Point 11
Point 10
Poin t
1
Point
9
Point
8
Point
7
P oin t
6
Point
5
Point
4
Point
3
Point 21
P oin t
2 0
Point
2
Point 19
Point 18
P oint
17
Point 16
Point 15
Point 14
Point 13
Point 1 2
Point 11
Point 10
Point
1
0.0010
0.0005
0.0000
Points
Sam
ple
Range
_R=0.000359
UCL=0.000924
LCL=0
Operator 1 Operator 2 Operator 3
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
P oin t
3
Poin
t 21
Poin
t 20
Point 2
Po in
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Po in
t 10
Point 1
Point 9
Point 8
P oint
7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Po in
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
Point 9
Point 8
Point 7
P oin t
6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Po in
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
0.11
0.09
0.07
Points
Sam
ple
Mean
__X=0.09606UCL=0.09643LCL=0.09570
Operator 1 Operator 2 Operator 3
Point 9
Point 8
Poi n
t 7
Poi n
t 6
Point 5
Point 4
Point 3
Point 2
1
Poi n
t 20
Point 2
Point 1
9
Point 1
8
Point 1
7
Poi n
t 16
Poi nt 1
5
Point 1
4
Point 1
3
Point 1
2
Poi n
t 11
Poi nt 1
0
Point 1
0.11
0.09
0.07
Points
Operator 3Operator 2Operator 1
0.11
0.09
0.07
Operator
Point 9
Point 8
Point 7
P oin t 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Po in
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
P oint
1
0.11
0.09
0.07
Points
Avera
ge Operator 1
Operator 2
Operator 3
Operator
Gage name: C ogniTens WLS 400M
Date of study : Nov 2011
Reported by : M. Hestness
Tolerance: ±.005
Misc: Precision Surface Block
Components of Variation
R Chart by Operator
Xbar Chart by Operator
Z Location by Points
Z Location by Operator
Points * Operator Interaction
Gage R&R (ANOVA) for Z Location
·a ; I:- I ;. :
•
62
Copyright 2011 Lockheed Martin
Gage R&R Study - ANOVA Method Gage R&R for Z Location
Gage name: CogniTens WLS 400M
Date of study: Nov 2011
Reported by: M. Hestness
Tolerance: ±.005
Misc: Precision Surface Block
Two-Way ANOVA Table With Interaction Source DF SS MS F P
Points 20 0.0156222 0.0007811 18321.8 0.000
Operator 2 0.0000021 0.0000010 24.4 0.000
Points * Operator 40 0.0000017 0.0000000 0.9 0.716
Repeatability 126 0.0000063 0.0000001
Total 188 0.0156323
Alpha to remove interaction term = 0.25
Two-Way ANOVA Table Without Interaction Source DF SS MS F P
Points 20 0.0156222 0.0007811 16178.4 0.000
Operator 2 0.0000021 0.0000010 21.5 0.000
Repeatability 166 0.0000080 0.0000000
Total 188 0.0156323
Gage R&R %Contribution
Source VarComp (of VarComp)
Total Gage R&R 0.0000001 0.07
Repeatability 0.0000000 0.06
Reproducibility 0.0000000 0.02
Operator 0.0000000 0.02
Part-To-Part 0.0000868 99.93
Total Variation 0.0000868 100.00
Process tolerance = 0.01
Study Var %Study Var %Tolerance
Source StdDev (SD) (6 * SD) (%SV) (SV/Toler)
Total Gage R&R 0.0002530 0.0015180 2.71 15.18
Repeatability 0.0002197 0.0013184 2.36 13.18
Reproducibility 0.0001254 0.0007524 1.35 7.52
Operator 0.0001254 0.0007524 1.35 7.52
Part-To-Part 0.0093158 0.0558949 99.96 558.95
Total Variation 0.0093193 0.0559155 100.00 559.16
Number of Distinct Categories = 51
63
Copyright 2011 Lockheed Martin
ATOS Triple Scan Input Data
Operator 1 Operator 2 Operator 3
Nominal Rep 1 Rep 2 Rep 3 Rep 1 Rep 2 Rep 3 Rep 1 Rep 2 Rep 3
0.1024 0.10486 0.10481 0.10482 0.10471 0.10472 0.10455 0.10494 0.10493 0.10482
0.0952 0.09681 0.09672 0.09684 0.09673 0.09678 0.09663 0.09689 0.09687 0.09681
0.0933 0.09444 0.09449 0.09441 0.09443 0.09442 0.09429 0.09443 0.09441 0.0944
0.0930 0.09394 0.09387 0.09374 0.09387 0.09392 0.09389 0.09389 0.09387 0.09386
0.0928 0.09340 0.09339 0.09327 0.09351 0.09338 0.09333 0.09338 0.09342 0.09339
0.0929 0.09308 0.09290 0.09275 0.09296 0.09295 0.09298 0.09296 0.09296 0.09303
0.0950 0.09482 0.09464 0.09449 0.09463 0.09455 0.09452 0.09451 0.09455 0.09466
0.0946 0.09573 0.09576 0.09562 0.09555 0.09565 0.09546 0.09568 0.0957 0.09555
0.0997 0.10029 0.10038 0.10028 0.10024 0.10027 0.10019 0.10022 0.10022 0.10012
0.1027 0.10303 0.10314 0.10295 0.10303 0.10301 0.10303 0.103 0.103 0.10295
0.1020 0.10221 0.10228 0.10202 0.10222 0.10232 0.10222 0.10223 0.10222 0.10227
0.0976 0.09744 0.09726 0.09721 0.09739 0.09732 0.09731 0.09737 0.09733 0.09739
0.0897 0.08960 0.08950 0.08916 0.08938 0.08945 0.08942 0.08933 0.08932 0.08941
0.0821 0.08177 0.08151 0.08138 0.08157 0.08157 0.08144 0.0815 0.08148 0.08166
0.1012 0.10173 0.10190 0.10179 0.10173 0.10175 0.10156 0.10175 0.10171 0.10158
0.1093 0.10926 0.10925 0.10912 0.10919 0.10924 0.1091 0.1092 0.10919 0.10907
0.1120 0.11161 0.11142 0.11146 0.11142 0.1116 0.11143 0.11159 0.1115 0.11151
0.1080 0.10753 0.10734 0.10713 0.10739 0.10751 0.10738 0.10737 0.10731 0.10732
0.0977 0.09701 0.09703 0.09694 0.09724 0.09722 0.09717 0.0971 0.09707 0.09714
0.0848 0.08365 0.08351 0.08321 0.08351 0.08359 0.0835 0.08338 0.08337 0.0835
0.0735 0.07230 0.07203 0.07171 0.07206 0.0722 0.0722 0.07198 0.07195 0.07216
64
Copyright 2011 Lockheed Martin
0.110.100.090.080.07
0.003
0.002
0.001
0.000
-0.001
-0.002
Reference Value
Bia
s
0
Regression
95% CI
Data
Avg Bias
BiasLinearity
3.0
1.5
0.0
Pe
rce
nt
C onstant -0.0028433 0.0006958 0.000
Slope 0.031101 0.007205 0.000
Predictor C oef SE C oef P
Gage Linearity
S 0.0008776 R-Sq 9.1%
Linearity 0.0017549 %Linearity 3.1
A v erage 0.0001476 0.3 0.000
0.0735 -0.0014344 2.5 0.000
0.0821 -0.0005578 1.0 0.000
0.0848 -0.0013311 2.4 0.000
0.0897 -0.0003033 0.5 0.000
0.0928 0.0005856 1.0 0.000
0.0929 0.0000522 0.1 0.187
0.093 0.0008722 1.5 0.000
Reference Bias %Bias P
Gage Bias
Gage name: A TO S Triple Scan
Date of study : Nov 2011
Reported by : M. Hestness
Tolerance: ±.005
Misc: Precision Surface Block
Percent of Process Variation
Gage Linearity and Bias Study for Measured
Part-to-PartReprodRepeatGage R&R
500
250
0
Perc
ent
% Contribution
% Study Var
% Tolerance
Point
9
Poin t
8
Point
7
Point
6
P oint
5
Point
4
Point
3
Point 21
P oin t
20
Point
2
Point 19
Point 18
Point 17
Point 16
Point 15
P oin t
1 4
Point 13
Point 12
Point 11
Point 10
Point
1
Point
9
Point
8
Point
7
Point
6
Poin t
5
Point
4
Point
3
Point 21
Point 2 0
Point
2
Point 19
P oin t
18
Point 17
Point 16
Point 15
Point 14
Point 13
Point 12
P oin t
1 1
Point 10
Point
1
P oin t
9
Point
8
Point
7
Point
6
Point
5
Point
4
Point
3
P oint
21
Point 20
Point
2
Point 19
Point 1 8
Point 17
Point 16
P oin t
15
Point 14
Point 13
Point 12
Point 11
Point 10
Point
1
0.00050
0.00025
0.00000
Points
Sam
ple
Range
_R=0.0001510
UCL=0.0003886
LCL=0
Operator 1 Operator 2 Operator 3
Point 9
P oin t
8
Point 7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Po in
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Po in
t 11
Poin
t 10
Point 1
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
Point 3
Po in
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Po in
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
P oin t
9
Point 8
Point 7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Po in
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Po in
t 12
Poin
t 11
Poin
t 10
Point 1
0.11
0.09
0.07
Points
Sam
ple
Mean
__X=0.09631UCL=0.09647LCL=0.09616
Operator 1 Operator 2 Operator 3
Point 9
Point 8
Poi n
t 7
Poi n
t 6
Point 5
Point 4
Point 3
Point 2
1
Poi n
t 20
Point 2
Point 1
9
Point 1
8
Point 1
7
Poi n
t 16
Poi nt 1
5
Point 1
4
Point 1
3
Point 1
2
Poi n
t 11
Poi nt 1
0
Point 1
0.11
0.09
0.07
Points
Operator 3Operator 2Operator 1
0.11
0.09
0.07
Operator
Point 9
Point 8
Point 7
P oin t 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Po in
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
P oint
1
0.11
0.09
0.07
Points
Avera
ge Operator 1
Operator 2
Operator 3
Operator
Gage name: A TO S Triple Scan
Date of study : Nov 2011
Reported by : M. Hestness
Tolerance: ±.005
Misc: Precision Surface Block
Components of Variation
R Chart by Operator
Xbar Chart by Operator
Z Location by Points
Z Location by Operator
Points * Operator Interaction
Gage R&R (ANOVA) for Z Location
I •
1 . J l l~ I . . _ EZ:SL:::::I I
~I
I! ! !I tv\r-1D
65
Copyright 2011 Lockheed Martin
Gage R&R Study - ANOVA Method Gage R&R for Z Location
Gage name: ATOS Triple Scan
Date of study: Nov 2011
Reported by: M. Hestness
Tolerance: ±.005
Misc: Precision Surface Block
Two-Way ANOVA Table With Interaction Source DF SS MS F P
Points 20 0.0159177 0.0007959 73536.4 0.000
Operator 2 0.0000000 0.0000000 0.0 0.975
Points * Operator 40 0.0000004 0.0000000 1.1 0.285
Repeatability 126 0.0000012 0.0000000
Total 188 0.0159193
Alpha to remove interaction term = 0.25
Two-Way ANOVA Table Without Interaction Source DF SS MS F P
Points 20 0.0159177 0.0007959 81253.4 0.000
Operator 2 0.0000000 0.0000000 0.0 0.973
Repeatability 166 0.0000016 0.0000000
Total 188 0.0159193
Gage R&R %Contribution
Source VarComp (of VarComp)
Total Gage R&R 0.0000000 0.01
Repeatability 0.0000000 0.01
Reproducibility 0.0000000 0.00
Operator 0.0000000 0.00
Part-To-Part 0.0000884 99.99
Total Variation 0.0000884 100.00
Process tolerance = 0.01
Study Var %Study Var %Tolerance
Source StdDev (SD) (6 * SD) (%SV) (SV/Toler)
Total Gage R&R 0.0000990 0.0005938 1.05 5.94
Repeatability 0.0000990 0.0005938 1.05 5.94
Reproducibility 0.0000000 0.0000000 0.00 0.00
Operator 0.0000000 0.0000000 0.00 0.00
Part-To-Part 0.0094038 0.0564226 99.99 564.23
Total Variation 0.0094043 0.0564257 100.00 564.26
Number of Distinct Categories = 133
66
Copyright 2011 Lockheed Martin
Rexcan 4 White Light Scanner Input Data
Operator 1 Operator 2 Operator 3
Nominal Rep 1 Rep 2 Rep 3 Rep 1 Rep 2 Rep 3 Rep 1 Rep 2 Rep 3
0.1024 0.10584 0.10367 0.10459 0.10485 0.10523 0.10419 0.10461 0.10473 0.10504
0.0952 0.09663 0.09694 0.0957 0.09679 0.09697 0.09635 0.09535 0.09567 0.09546
0.0933 0.09531 0.09493 0.09314 0.09474 0.09439 0.09534 0.09414 0.0944 0.09318
0.0930 0.09549 0.09431 0.09455 0.09374 0.09438 0.09617 0.09386 0.09339 0.09374
0.0928 0.09311 0.09258 0.09343 0.09383 0.09466 0.09366 0.09304 0.09307 0.09422
0.0929 0.09319 0.09289 0.09265 0.09329 0.09269 0.09376 0.09334 0.09243 0.09315
0.0950 0.09521 0.09758 0.09715 0.0959 0.09611 0.09657 0.09472 0.09536 0.09609
0.0946 0.09587 0.09723 0.09543 0.09589 0.09725 0.09646 0.09624 0.09498 0.09728
0.0997 0.10198 0.10023 0.10052 0.10058 0.10029 0.09976 0.09906 0.09887 0.09936
0.1027 0.10306 0.10484 0.1035 0.10253 0.10265 0.1026 0.10367 0.1053 0.10415
0.1020 0.10241 0.10090 0.10032 0.10135 0.10157 0.10012 0.10287 0.10077 0.10187
0.0976 0.09689 0.09814 0.0971 0.09821 0.0964 0.09579 0.09683 0.09798 0.09836
0.0897 0.09109 0.09091 0.08946 0.09147 0.09005 0.08938 0.08861 0.09004 0.09077
0.0821 0.08236 0.08092 0.08083 0.08406 0.08156 0.08365 0.08161 0.08111 0.08221
0.1012 0.10287 0.10142 0.10263 0.10017 0.09902 0.10204 0.09981 0.10064 0.1019
0.1093 0.10935 0.10780 0.10888 0.10914 0.10938 0.10751 0.10932 0.1091 0.10974
0.1120 0.11077 0.11071 0.11262 0.11064 0.1088 0.10918 0.11137 0.11036 0.11182
0.1080 0.10782 0.10659 0.10819 0.10719 0.10603 0.10666 0.10739 0.10715 0.10651
0.0977 0.09702 0.09778 0.09778 0.09802 0.09783 0.09648 0.09827 0.09701 0.09747
0.0848 0.08364 0.08287 0.08398 0.08486 0.08354 0.0832 0.08343 0.08353 0.08472
0.0735 0.07260 0.07343 0.07298 0.07628 0.07159 0.07136 0.07306 0.07237 0.07391
67
Copyright 2011 Lockheed Martin
0.110.100.090.080.07
0.003
0.002
0.001
0.000
-0.001
-0.002
-0.003
-0.004
Reference Value
Bia
s 0
Regression
95% CI
Data
Avg Bias
BiasLinearity
1.0
0.5
0.0
Pe
rce
nt
C onstant 0.001189 0.001009 0.240
Slope -0.00945 0.01045 0.367
Predictor C oef SE C oef P
Gage Linearity
S 0.0012727 R-Sq 0.4%
Linearity 0.0005149 %Linearity 0.9
A v erage 0.0002808 0.5 0.000
0.0735 -0.0004356 0.8 0.441
0.0821 -0.0000656 0.1 0.857
0.0848 -0.0010478 1.9 0.002
0.0897 0.0004978 0.9 0.152
0.0928 0.0007111 1.3 0.016
0.0929 0.0001433 0.3 0.353
0.093 0.0014033 2.6 0.002
Reference Bias %Bias P
Gage Bias
Gage name: Rexcan 4 White Light Scanner
Date of study : Nov 2011
Reported by : M. Hestness
Tolerance: ±.005
Misc: Precision Surface Block
Percent of Process Variation
Gage Linearity and Bias Study for Measured
Part-to-PartReprodRepeatGage R&R
400
200
0
Perc
ent
% Contribution
% Study Var
% Tolerance
Point
9
Point
8
Point
7
Poin t
6
Point
5
Point
4
Point
3
Point 21
Point 20
Poin t
2
Point 1 9
Point 18
Point 17
Point 16
Point 15
P oint
14
Point 1 3
Point 12
Point 11
Point 10
Point
1
Poin t
9
Point
8
Point
7
Point
6
Point
5
Point
4
Poin t
3
Point 2 1
Point 20
Point
2
Point 19
Point 18
P oint
17
Point 1 6
Point 15
Point 14
Point 13
Point 12
P oint
11
Point 1 0
Point
1
Point
9
Point
8
Point
7
Poin t
6
Point
5
Point
4
Point
3
Point 21
Point 20
Poin t
2
Point 1 9
Point 18
Point 17
Point 16
Point 15
P oint
14
Point 1 3
Point 12
Point 11
Point 10
Point
1
0.004
0.002
0.000
Points
Sam
ple
Range
_R=0.001468
UCL=0.003779
LCL=0
Operator 1 Operator 2 Operator 3
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
0.11
0.09
0.07
Points
Sam
ple
Mean
__X=0.09645UCL=0.09795LCL=0.09495
Operator 1 Operator 2 Operator 3
Point 9
Point 8
Poi n
t 7
Poi n
t 6
Point 5
Point 4
Point 3
Point 2
1
Poi n
t 20
Point 2
Point 1
9
Point 1
8
Point 1
7
Poi n
t 16
Poi nt 1
5
Point 1
4
Point 1
3
Point 1
2
Poi n
t 11
Poi nt 1
0
Point 1
0.11
0.09
0.07
Points
Operator 3Operator 2Operator 1
0.11
0.09
0.07
Operator
Point 9
Point 8
Point 7
P oin t 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Po in
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
P oint
1
0.11
0.09
0.07
Points
Avera
ge Operator 1
Operator 2
Operator 3
Operator
Gage name: Rexcan 4 White Light Scanner
Date of study : Nov 2011
Reported by : M. Hestness
Tolerance: ±.005
Misc: Precision Surface Block
Components of Variation
R Chart by Operator
Xbar Chart by Operator
Z Location by Points
Z Location by Operator
Points * Operator Interaction
Gage R&R (ANOVA) for Z Location
• •
• •
D
I ~ ~ o= JIL::.....--....11~ I r::::z:::s/:::::1
~I
I! t !I r~o
68
Copyright 2011 Lockheed Martin
Gage R&R Study - ANOVA Method Gage R&R for Z Location
Gage name: Rexcan 4 White Light Scanner
Date of study: Nov 2011
Reported by: M. Hestness
Tolerance: ±.005
Misc: Precision Surface Block
Two-Way ANOVA Table With Interaction Source DF SS MS F P
Points 20 0.0147228 0.0007361 705.552 0.000
Operator 2 0.0000018 0.0000009 0.881 0.422
Points * Operator 40 0.0000417 0.0000010 1.376 0.094
Repeatability 126 0.0000956 0.0000008
Total 188 0.0148619
Alpha to remove interaction term = 0.25
Gage R&R %Contribution
Source VarComp (of VarComp)
Total Gage R&R 0.0000009 1.03
Repeatability 0.0000008 0.92
Reproducibility 0.0000001 0.12
Operator 0.0000000 0.00
Operator*Points 0.0000001 0.12
Part-To-Part 0.0000817 98.97
Total Variation 0.0000825 100.00
Process tolerance = 0.01
Study Var %Study Var %Tolerance
Source StdDev (SD) (6 * SD) (%SV) (SV/Toler)
Total Gage R&R 0.0009238 0.0055427 10.17 55.43
Repeatability 0.0008708 0.0052251 9.59 52.25
Reproducibility 0.0003082 0.0018492 3.39 18.49
Operator 0.0000000 0.0000000 0.00 0.00
Operator*Points 0.0003082 0.0018492 3.39 18.49
Part-To-Part 0.0090375 0.0542253 99.48 542.25
Total Variation 0.0090846 0.0545078 100.00 545.08
Number of Distinct Categories = 13
69
Copyright 2011 Lockheed Martin
Appendix G: Input Data and Results for Spot Face Panel
CogniTens WLS 400M Input Data
Operator 1 Operator 2 Operator 3
Nominal Rep 1 Rep 2 Rep 3 Rep 1 Rep 2 Rep 3 Rep 1 Rep 2 Rep 3
-0.0006 -0.0003 -0.0012 -0.0009 -0.0023 0.0006 -0.0009 -0.0007 -0.0021 0.0012 -0.0011 -0.0006 -0.0008 -0.0005 -0.0012 -0.0005 -0.0014 -0.0011 0.0000 -0.0017 -0.0037 -0.0041 -0.0028 -0.0028 -0.0041 -0.0052 -0.0031 -0.0030 -0.0033 -0.0030 -0.0077 -0.0079 -0.0071 -0.0070 -0.0072 -0.0074 -0.0080 -0.0067 -0.0065 -0.0058 -0.0115 -0.0108 -0.0111 -0.0110 -0.0118 -0.0113 -0.0117 -0.0112 -0.0126 -0.0112 -0.0155 -0.0146 -0.0146 -0.0150 -0.0160 -0.0148 -0.0143 -0.0161 -0.0157 -0.0156 -0.0194 -0.0196 -0.0189 -0.0190 -0.0201 -0.0196 -0.0192 -0.0200 -0.0193 -0.0197 -0.0009 -0.0018 -0.0012 -0.0009 -0.0007 -0.0010 -0.0009 -0.0014 -0.0017 -0.0009 -0.0016 -0.0017 -0.0014 -0.0024 -0.0021 -0.0023 -0.0021 -0.0014 -0.0018 -0.0014 -0.0035 -0.0033 -0.0040 -0.0039 -0.0045 -0.0052 -0.0039 -0.0030 -0.0037 -0.0035 -0.0076 -0.0078 -0.0075 -0.0079 -0.0069 -0.0069 -0.0073 -0.0076 -0.0074 -0.0072 -0.0107 -0.0106 -0.0110 -0.0112 -0.0107 -0.0109 -0.0102 -0.0112 -0.0105 -0.0103 -0.0156 -0.0157 -0.0155 -0.0159 -0.0160 -0.0146 -0.0159 -0.0156 -0.0156 -0.0157 -0.0194 -0.0204 -0.0194 -0.0189 -0.0194 -0.0187 -0.0192 -0.0184 -0.0190 -0.0198 -0.0014 -0.0011 -0.0013 -0.0011 -0.0015 -0.0016 -0.0013 -0.0014 -0.0010 -0.0013 -0.0019 -0.0016 -0.0014 -0.0018 -0.0017 -0.0016 -0.0015 -0.0017 -0.0015 -0.0013 -0.0034 -0.0032 -0.0034 -0.0034 -0.0034 -0.0036 -0.0034 -0.0031 -0.0032 -0.0035 -0.0075 -0.0075 -0.0077 -0.0074 -0.0078 -0.0077 -0.0074 -0.0076 -0.0073 -0.0078 -0.0118 -0.0115 -0.0113 -0.0117 -0.0117 -0.0114 -0.0116 -0.0117 -0.0117 -0.0117 -0.0158 -0.0157 -0.0156 -0.0155 -0.0155 -0.0157 -0.0156 -0.0156 -0.0155 -0.0157 -0.0190 -0.0186 -0.0188 -0.0188 -0.0189 -0.0189 -0.0187 -0.0188 -0.0192 -0.0190
70
Copyright 2011 Lockheed Martin
0.000-0.005-0.010-0.015-0.020
0.002
0.001
0.000
-0.001
-0.002
Reference Value
Bia
s
0
Regression
95% CI
Data
Avg Bias
BiasLinearity
0.8
0.4
0.0
Pe
rce
nt
C onstant 0.00002382 0.00006249 0.704
Slope -0.007155 0.005781 0.217
Predictor C oef SE C oef P
Gage Linearity
S 0.0005253 R-Sq 0.8%
Linearity 0.0002896 %Linearity 0.7
A v erage 0.0000850 0.2 0.010
-0.019425 -0.0000552 0.1 0.668
-0.0194 0.0001507 0.4 0.512
-0.019 0.0001712 0.4 0.028
-0.015775 0.0001752 0.4 0.001
-0.015625 -0.0000030 0.0 0.985
-0.01545 0.0002840 0.7 0.189
-0.0118 0.0002201 0.5 0.004
Reference Bias %Bias P
Gage Bias
Gage name: C ogniTens WLS 400M
Date of study : Nov 2011
Reported by : M. Hestness
Tolerance: -.020 to .000
Misc: Spot F ace Panel
Percent of Process Variation
Gage Linearity and Bias Study for Measured
Part-to-PartReprodRepeatGage R&R
200
100
0
Perc
ent
% Contribution
% Study Var
% Tolerance
Point
9
Point
8
Point
7
Point
6
Point
5
Point
4
Poin t
3
Point 21
Point 20
Point
2
Point 19
Point 1 8
Point 17
Point 16
Point 15
P oin t
14
Point 13
Point 12
Point 11
Point 10
Point
1
Point
9
Point
8
P oint
7
Point
6
Point
5
Point
4
Point
3
P oin t
21
Point 20
Point
2
Point 19
Point 18
Point 1 7
Point 16
Point 15
Point 14
Point 1 3
Point 12
Point 11
Point 10
Poin t
1
Point
9
Point
8
Point
7
P oin t
6
Point
5
Point
4
Point
3
Point 21
P oin t
2 0
Point
2
Point 19
Point 18
P oint
17
Point 16
Point 15
Point 14
Point 13
Point 1 2
Point 11
Point 10
Point
1
0.0030
0.0015
0.0000
Points
Sam
ple
Range
_R=0.000714
UCL=0.001838
LCL=0
Operator 1 Operator 2 Operator 3
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
P oin t
3
Poin
t 21
Poin
t 20
Point 2
Po in
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Po in
t 10
Point 1
Point 9
Point 8
P oint
7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Po in
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
Point 9
Point 8
Point 7
P oin t
6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Po in
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
0.00
-0.01
-0.02
Points
Sam
ple
Mean
__X=-0.00847UCL=-0.00774LCL=-0.00920
Operator 1 Operator 2 Operator 3
Point 9
Point 8
Point 7
Poin t 6
Point
5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Po in
t 12
Poin
t 11
Poin
t 10
Point 1
0.00
-0.01
-0.02
Points
Operator 3Operator 2Operator 1
0.00
-0.01
-0.02
Operator
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
0.00
-0.01
-0.02
Points
Avera
ge Operator 1
Operator 2
Operator 3
Operator
Gage name: C ogniTens WLS 400M
Date of study : Nov 2011
Reported by : M. Hestness
Tolerance: -.020 to .000
Misc: Spot F ace Panel
Components of Variation
R Chart by Operator
Xbar Chart by Operator
Spot Face Depth by Points
Spot Face Depth by Operator
Points * Operator Interaction
Gage R&R (ANOVA) for Spot Face Depth
.. D • • • • • • I • I
• • • I ' . . -· .. t=t--~F-t-+4 • • • ' !I • •• • • • •
• • • •
I = = . ~ ] 1~ I 1Sl:3/Sll
li*e~l 1 ~ ~ $ 1
1~1 ~D
71
Copyright 2011 Lockheed Martin
Gage R&R Study - ANOVA Method Gage R&R for Spot Face Depth
Gage name: CogniTens WLS 400M
Date of study: Nov 2011
Reported by: M. Hestness
Tolerance: -.020 to .000
Misc: Spot Face Panel
Two-Way ANOVA Table With Interaction Source DF SS MS F P
Points 20 0.0081490 0.0004075 1331.76 0.000
Operator 2 0.0000007 0.0000003 1.13 0.333
Points * Operator 40 0.0000122 0.0000003 1.29 0.148
Repeatability 126 0.0000300 0.0000002
Total 188 0.0081919
Alpha to remove interaction term = 0.25
Gage R&R %Contribution
Source VarComp (of VarComp)
Total Gage R&R 0.0000003 0.57
Repeatability 0.0000002 0.52
Reproducibility 0.0000000 0.05
Operator 0.0000000 0.00
Operator*Points 0.0000000 0.05
Part-To-Part 0.0000452 99.43
Total Variation 0.0000455 100.00
Process tolerance = 0.02
Study Var %Study Var %Tolerance
Source StdDev (SD) (6 * SD) (%SV) (SV/Toler)
Total Gage R&R 0.0005110 0.0030659 7.58 15.33
Repeatability 0.0004876 0.0029255 7.23 14.63
Reproducibility 0.0001529 0.0009173 2.27 4.59
Operator 0.0000252 0.0001510 0.37 0.75
Operator*Points 0.0001508 0.0009048 2.24 4.52
Part-To-Part 0.0067260 0.0403557 99.71 201.78
Total Variation 0.0067453 0.0404720 100.00 202.36
Number of Distinct Categories = 18
72
Copyright 2011 Lockheed Martin
ATOS Triple Scan Input Data
Operator 1 Operator 2 Operator 3
Nominal Rep 1 Rep 2 Rep 3 Rep 1 Rep 2 Rep 3 Rep 1 Rep 2 Rep 3
-0.0006 -0.0006 -0.0006 -0.0007 -0.0006 -0.0005 -0.0006 -0.0006 -0.0007 -0.0007 -0.0011 -0.0013 -0.0014 -0.0011 -0.0012 -0.001 -0.0012 -0.0012 -0.0012 -0.0011 -0.0037 -0.0043 -0.0044 -0.0039 -0.0039 -0.004 -0.0038 -0.0037 -0.0036 -0.0038 -0.0077 -0.0075 -0.0075 -0.0079 -0.0079 -0.0076 -0.0078 -0.0074 -0.0074 -0.0075 -0.0115 -0.0113 -0.0117 -0.012 -0.012 -0.0125 -0.0116 -0.0114 -0.0116 -0.0115 -0.0155 -0.015 -0.0184 -0.0167 -0.0156 -0.0157 -0.0156 -0.0156 -0.0155 -0.0153 -0.0194 -0.0206 -0.0201 -0.0204 -0.0199 -0.0199 -0.0197 -0.0201 -0.0199 -0.0201 -0.0009 -0.0008 -0.0007 -0.0007 -0.0009 -0.0007 -0.0009 -0.0008 -0.0008 -0.0008 -0.0016 -0.0017 -0.0021 -0.0018 -0.002 -0.0019 -0.002 -0.0018 -0.0018 -0.0018 -0.0035 -0.0042 -0.004 -0.0048 -0.0037 -0.0038 -0.0038 -0.0038 -0.0037 -0.0038 -0.0076 -0.0078 -0.008 -0.0079 -0.0079 -0.0078 -0.0077 -0.0077 -0.0077 -0.0077 -0.0107 -0.0107 -0.011 -0.0115 -0.0107 -0.0107 -0.0108 -0.0107 -0.0107 -0.0107 -0.0156 -0.0157 -0.0163 -0.0157 -0.0154 -0.0155 -0.0156 -0.0156 -0.0156 -0.0156 -0.0194 -0.02 -0.0202 -0.0199 -0.0196 -0.0197 -0.0196 -0.0195 -0.0187 -0.0168 -0.0014 -0.0012 -0.0011 -0.0012 -0.0011 -0.0011 -0.0011 -0.0011 -0.0011 -0.0011 -0.0019 -0.0016 -0.0016 -0.0016 -0.0015 -0.0015 -0.0015 -0.0015 -0.0014 -0.0015 -0.0034 -0.0035 -0.0033 -0.0034 -0.0034 -0.0034 -0.0034 -0.0034 -0.0034 -0.0034 -0.0075 -0.0076 -0.0076 -0.0076 -0.0076 -0.0076 -0.0076 -0.0076 -0.0076 -0.0076 -0.0118 -0.0119 -0.0118 -0.0118 -0.0118 -0.0118 -0.0118 -0.0119 -0.0118 -0.0118 -0.0158 -0.0158 -0.0158 -0.0158 -0.0157 -0.0158 -0.0158 -0.0157 -0.0158 -0.0158 -0.0190 -0.0192 -0.0192 -0.0192 -0.0192 -0.0192 -0.0192 -0.0191 -0.0193 -0.0192
73
Copyright 2011 Lockheed Martin
0.000-0.005-0.010-0.015-0.020
0.003
0.002
0.001
0.000
-0.001
-0.002
-0.003
Reference Value
Bia
s 0
Regression
95% CI
Data
Avg Bias
BiasLinearity
1.0
0.5
0.0
Pe
rce
nt
C onstant 0.00001275 0.00004894 0.795
Slope 0.013458 0.004527 0.003
Predictor C oef SE C oef P
Gage Linearity
S 0.0004114 R-Sq 4.5%
Linearity 0.0005553 %Linearity 1.3
A v erage -0.0001024 0.2 0.000
-0.019425 -0.0006528 1.6 0.000
-0.0194 0.0000667 0.2 0.861
-0.019 -0.0002000 0.5 0.000
-0.015775 -0.0000028 0.0 0.805
-0.015625 -0.0000417 0.1 0.682
-0.01545 -0.0004833 1.2 0.231
-0.0118 -0.0000222 0.1 0.079
Reference Bias %Bias P
Gage Bias
Gage name: A TO S Triple Scan
Date of study : Nov 2011
Reported by : M. Hestness
Tolerance: -.020 to .000
Misc: Spot F ace Panel
Percent of Process Variation
Gage Linearity and Bias Study for Measured
Part-to-PartReprodRepeatGage R&R
200
100
0
Perc
ent
% Contribution
% Study Var
% Tolerance
Point
9
Point
8
Point
7
Point
6
Point
5
Point
4
Poin t
3
Point 21
Point 20
Point
2
Point 19
Point 1 8
Point 17
Point 16
Point 15
P oin t
14
Point 13
Point 12
Point 11
Point 10
Point
1
Point
9
Point
8
P oint
7
Point
6
Point
5
Point
4
Point
3
P oin t
21
Point 20
Point
2
Point 19
Point 18
Point 1 7
Point 16
Point 15
Point 14
Point 1 3
Point 12
Point 11
Point 10
Poin t
1
Point
9
Point
8
Point
7
P oin t
6
Point
5
Point
4
Point
3
Point 21
P oin t
2 0
Point
2
Point 19
Point 18
P oint
17
Point 16
Point 15
Point 14
Point 13
Point 1 2
Point 11
Point 10
Point
1
0.0030
0.0015
0.0000
Points
Sam
ple
Range
_R=0.000271UCL=0.000699LCL=0
Operator 1 Operator 2 Operator 3
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
P oin t
3
Poin
t 21
Poin
t 20
Point 2
Po in
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Po in
t 10
Point 1
Point 9
Point 8
P oint
7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Po in
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
Point 9
Point 8
Point 7
P oin t
6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Po in
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
0.00
-0.01
-0.02
Points
Sam
ple
Mean
__X=-0.00866UCL=-0.00838LCL=-0.00893
Operator 1 Operator 2 Operator 3
Point 9
Point 8
Point 7
Poin t 6
Point
5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Po in
t 12
Poin
t 11
Poin
t 10
Point 1
0.00
-0.01
-0.02
Points
Operator 3Operator 2Operator 1
0.00
-0.01
-0.02
Operator
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
0.00
-0.01
-0.02
Points
Avera
ge Operator 1
Operator 2
Operator 3
Operator
Gage name: A TO S Triple Scan
Date of study : Nov 2011
Reported by : M. Hestness
Tolerance: -.020 to .000
Misc: Spot F ace Panel
Components of Variation
R Chart by Operator
Xbar Chart by Operator
Spot Face Depth by Points
Spot Face Depth by Operator
Points * Operator Interaction
Gage R&R (ANOVA) for Spot Face Depth
• D •
t=i=-A~<::4-- ·~ I : · 11 • • • • • • •
•
I .= .= . ~ 1 1~ I I'SL3/S[I
1~1
74
Copyright 2011 Lockheed Martin
Gage R&R Study - ANOVA Method Gage R&R for Spot Face Depth
Gage name: ATOS Triple Scan
Date of study: Nov 2011
Reported by: M. Hestness
Tolerance: -.020 to .000
Misc: Spot Face Panel
Two-Way ANOVA Table With Interaction Source DF SS MS F P
Points 20 0.0084910 0.0004246 2197.22 0.000
Operator 2 0.0000025 0.0000012 6.42 0.004
Points * Operator 40 0.0000077 0.0000002 1.99 0.002
Repeatability 126 0.0000122 0.0000001
Total 188 0.0085135
Alpha to remove interaction term = 0.25
Gage R&R %Contribution
Source VarComp (of VarComp)
Total Gage R&R 0.0000001 0.31
Repeatability 0.0000001 0.21
Reproducibility 0.0000000 0.10
Operator 0.0000000 0.04
Operator*Points 0.0000000 0.07
Part-To-Part 0.0000472 99.69
Total Variation 0.0000473 100.00
Process tolerance = 0.02
Study Var %Study Var %Tolerance
Source StdDev (SD) (6 * SD) (%SV) (SV/Toler)
Total Gage R&R 0.0003817 0.0022905 5.55 11.45
Repeatability 0.0003115 0.0018690 4.53 9.35
Reproducibility 0.0002207 0.0013240 3.21 6.62
Operator 0.0001290 0.0007738 1.88 3.87
Operator*Points 0.0001791 0.0010743 2.60 5.37
Part-To-Part 0.0068667 0.0412000 99.85 206.00
Total Variation 0.0068773 0.0412636 100.00 206.32
Number of Distinct Categories = 25
75
Copyright 2011 Lockheed Martin
Rexcan 4 White Light Scanner Input Data
Operator 1 Operator 2 Operator 3
Nominal Rep 1 Rep 2 Rep 3 Rep 1 Rep 2 Rep 3 Rep 1 Rep 2 Rep 3
-0.0006 -0.0006 -0.0003 -0.0002 -0.0008 -0.0003 -0.0003 -0.0006 -0.0004 -0.0009 -0.0011 -0.0015 -0.0015 -0.0004 -0.0015 -0.0017 -0.0014 -0.0014 -0.0004 -0.0011 -0.0037 -0.0041 -0.0047 -0.0026 -0.0039 -0.0037 -0.0041 -0.0035 -0.0012 -0.0039 -0.0077 -0.0078 -0.0075 -0.0025 -0.0074 -0.0073 -0.0075 -0.0073 -0.0018 -0.008 -0.0115 -0.0118 -0.0116 -0.0041 -0.0113 -0.0114 -0.0113 -0.0121 -0.0037 -0.0117 -0.0155 -0.015 -0.015 -0.007 -0.014 -0.0135 -0.0136 -0.013 -0.0029 -0.0151 -0.0194 -0.0197 -0.0198 -0.0074 -0.0202 -0.02 -0.0193 -0.0198 -0.0085 -0.0196 -0.0009 -0.0013 -0.0001 -0.0003 -0.0012 -0.001 -0.0003 -0.001 -0.0014 -0.0021 -0.0016 -0.0019 -0.002 -0.0005 -0.0014 -0.0014 -0.0017 -0.0019 0 -0.0025 -0.0035 -0.0038 -0.0041 -0.0001 -0.0039 -0.0036 -0.0039 -0.0037 -0.0009 -0.0045 -0.0076 -0.0081 -0.008 -0.0003 -0.0078 -0.0074 -0.0075 -0.0076 -0.001 -0.0083 -0.0107 -0.0108 -0.01 -0.0039 -0.0107 -0.0111 -0.0106 -0.0108 -0.0012 -0.0111 -0.0156 -0.0154 -0.0153 -0.0002 -0.0161 -0.0152 -0.0158 -0.0141 -0.0012 -0.0158 -0.0194 -0.02 -0.0197 -0.0065 -0.0196 -0.0193 -0.0202 -0.019 -0.0023 -0.0189 -0.0014 -0.0012 -0.0013 -0.0014 -0.0015 -0.002 -0.0024 -0.0022 -0.0009 -0.0012 -0.0019 -0.0016 -0.0015 -0.0014 -0.0015 -0.0017 -0.0017 -0.0015 -0.0018 -0.0015 -0.0034 -0.0034 -0.0035 -0.0035 -0.0032 -0.0035 -0.0036 -0.0036 -0.0033 -0.0033 -0.0075 -0.0106 -0.0078 -0.0065 -0.009 -0.0084 -0.0104 -0.009 -0.0079 -0.0075 -0.0118 -0.0118 -0.0117 -0.0119 -0.0119 -0.0117 -0.0119 -0.0119 -0.0118 -0.0118 -0.0158 -0.0157 -0.0156 -0.0156 -0.0156 -0.0159 -0.0159 -0.0152 -0.0159 -0.0157 -0.0190 -0.0186 -0.0192 -0.0193 -0.0192 -0.0192 -0.0191 -0.0191 -0.019 -0.019
76
Copyright 2011 Lockheed Martin
0.000-0.005-0.010-0.015-0.020
0.020
0.015
0.010
0.005
0.000
Reference Value
Bia
s
0
Regression
95% CI
Data
Avg Bias
BiasLinearity
10
5
0
Pe
rce
nt
C onstant -0.0001664 0.0003524 0.637
Slope -0.12061 0.03260 0.000
Predictor C oef SE C oef P
Gage Linearity
S 0.0029625 R-Sq 6.8%
Linearity 0.0048214 %Linearity 12.1
A v erage 0.0008653 2.2 0.000
-0.019425 0.0022806 5.7 0.144
-0.0194 0.0032333 8.1 0.139
-0.019 -0.0000778 0.2 0.339
-0.015775 0.0000972 0.2 0.241
-0.015625 0.0035028 8.8 0.081
-0.01545 0.0033278 8.3 0.040
-0.0118 -0.0000222 0.1 0.339
Reference Bias %Bias P
Gage Bias
Gage name: Rexcan 4 White Light Sy stem
Date of study : Nov 2011
Reported by : M. Hestness
Tolerance: -.020 to .000
Misc: Spot F ace Panel
Percent of Process Variation
Gage Linearity and Bias Study for Measured
Part-to-PartReprodRepeatGage R&R
200
100
0
Perc
ent
% Contribution
% Study Var
% Tolerance
Point
9
Point
8
Point
7
Poin t
6
Point
5
Point
4
Point
3
Point 21
Point 20
Poin t
2
Point 1 9
Point 18
Point 17
Point 16
Point 15
P oint
14
Point 1 3
Point 12
Point 11
Point 10
Point
1
Poin t
9
Point
8
Point
7
Point
6
Point
5
Point
4
Poin t
3
Point 2 1
Point 20
Point
2
Point 19
Point 18
P oint
17
Point 1 6
Point 15
Point 14
Point 13
Point 12
P oint
11
Point 1 0
Point
1
Point
9
Point
8
Point
7
Poin t
6
Point
5
Point
4
Point
3
Point 21
Point 20
Poin t
2
Point 1 9
Point 18
Point 17
Point 16
Point 15
P oint
14
Point 1 3
Point 12
Point 11
Point 10
Point
1
0.016
0.008
0.000
Points
Sam
ple
Range
_R=0.00327
UCL=0.00843
LCL=0
Operator 1 Operator 2 Operator 3
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
0.00
-0.01
-0.02
Points
Sam
ple
Mean
__X=-0.00769UCL=-0.00434
LCL=-0.01104
Operator 1 Operator 2 Operator 3
Point 9
Point 8
Point 7
Poin t 6
Point
5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Po in
t 12
Poin
t 11
Poin
t 10
Point 1
0.00
-0.01
-0.02
Points
Operator 3Operator 2Operator 1
0.00
-0.01
-0.02
Operator
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
0.00
-0.01
-0.02
Points
Avera
ge Operator 1
Operator 2
Operator 3
Operator
Gage name: Rexcan 4 White Light Scanner
Date of study : Nov 2011
Reported by : M. Hestness
Tolerance: -020 to .000
Misc: Spot F ace Panel
Components of Variation
R Chart by Operator
Xbar Chart by Operator
Spot Face Depth by Points
Spot Face Depth by Operator
Points * Operator Interaction
Gage R&R (ANOVA) for Spot Face Depth
77
Copyright 2011 Lockheed Martin
Gage R&R Study - ANOVA Method Gage R&R for Spot Face Depth
Gage name: Rexcan 4 White Light Scanner
Date of study: Nov 2011
Reported by: M. Hestness
Tolerance: -020 to .000
Misc: Spot Face Panel
Two-Way ANOVA Table With Interaction Source DF SS MS F P
Points 20 0.0065752 0.0003288 98.2336 0.000
Operator 2 0.0000841 0.0000420 12.5591 0.000
Points * Operator 40 0.0001339 0.0000033 0.3416 1.000
Repeatability 126 0.0012343 0.0000098
Total 188 0.0080275
Alpha to remove interaction term = 0.25
Two-Way ANOVA Table Without Interaction Source DF SS MS F P
Points 20 0.0065752 0.0003288 39.8891 0.000
Operator 2 0.0000841 0.0000420 5.0998 0.007
Repeatability 166 0.0013682 0.0000082
Total 188 0.0080275
Gage R&R %Contribution
Source VarComp (of VarComp)
Total Gage R&R 0.0000088 19.77
Repeatability 0.0000082 18.57
Reproducibility 0.0000005 1.21
Operator 0.0000005 1.21
Part-To-Part 0.0000356 80.23
Total Variation 0.0000444 100.00
Process tolerance = 0.02
Study Var %Study Var %Tolerance
Source StdDev (SD) (6 * SD) (%SV) (SV/Toler)
Total Gage R&R 0.0029628 0.0177769 44.47 88.88
Repeatability 0.0028709 0.0172252 43.09 86.13
Reproducibility 0.0007324 0.0043942 10.99 21.97
Operator 0.0007324 0.0043942 10.99 21.97
Part-To-Part 0.0059677 0.0358061 89.57 179.03
Total Variation 0.0066627 0.0399762 100.00 199.88
Number of Distinct Categories = 2
78
Copyright 2011 Lockheed Martin
Appendix H: Input Data and Results for Countersink Panel
CogniTens WLS 400M Input Data
Operator 1 Operator 2 Operator 3
Nominal Diam. Rep 1 Rep 2 Rep 3 Rep 1 Rep 2 Rep 3 Rep 1 Rep 2 Rep 3
0.3764 0.3903 0.3878 0.3850 0.3846 0.3901 0.3886 0.3867 0.3920 0.3889
0.507 0.4455 0.4465 0.4441 0.4431 0.4465 0.4457 0.4462 0.4467 0.4475
0.5781 0.5742 0.5740 0.5720 0.5701 0.5744 0.5739 0.5733 0.5739 0.5752
0.401 0.3985 0.3971 0.3956 0.3944 0.3983 0.3965 0.3944 0.3972 0.3976
0.5286 0.5246 0.5237 0.5205 0.5222 0.5242 0.5208 0.5190 0.5216 0.5234
0.6403 0.6492 0.6490 0.6462 0.6448 0.6492 0.6451 0.6454 0.6472 0.6474
0.7778 0.7777 0.7790 0.7755 0.7741 0.7767 0.7735 0.7772 0.7760 0.7730
0.3709 0.3807 0.3780 0.3780 0.3806 0.3810 0.3806 0.3793 0.3817 0.3812
0.4443 0.4484 0.4457 0.4461 0.4466 0.4453 0.4465 0.4457 0.4443 0.4423
0.5726 0.6444 0.6431 0.6434 0.6428 0.6425 0.6431 0.6436 0.6449 0.6440
0.3954 0.3935 0.3815 0.3908 0.3905 0.3897 0.3890 0.3872 0.3906 0.3909
0.5199 0.5262 0.5235 0.5221 0.5223 0.5223 0.5210 0.5224 0.5231 0.5134
0.6399 0.6486 0.6464 0.6439 0.6454 0.6462 0.6440 0.6465 0.6447 0.6463
0.7756 0.7819 0.7776 0.7770 0.7778 0.7767 0.7743 0.7802 0.7783 0.7770
0.374 0.3706 0.3651 0.3681 0.3697 0.3685 0.3675 0.3675 0.3688 0.3709
0.4495 0.4407 0.4367 0.4375 0.4373 0.4351 0.4376 0.4359 0.4389 0.4381
0.5692 0.6420 0.6385 0.6388 0.6395 0.6358 0.6374 0.6381 0.6397 0.6385
0.396 0.3871 0.3820 0.3828 0.3840 0.3824 0.3815 0.3826 0.3841 0.3826
0.5218 0.5175 0.5215 0.5151 0.5209 0.5188 0.5192 0.5195 0.5220 0.5204
0.6464 0.6478 0.6463 0.6460 0.6439 0.6436 0.6437 0.6440 0.6469 0.6444
0.7747 0.7820 0.7766 0.7698 0.7760 0.7677 0.7716 0.7707 0.7729 0.7737
79
Copyright 2011 Lockheed Martin
0.80.70.60.50.4
0.075
0.050
0.025
0.000
-0.025
-0.050
Reference Value
Bia
s
0
Regression
95% CI
Data
Avg Bias
BiasLinearity
2
1
0
Pe
rce
nt
C onstant -0.010525 0.007934 0.186
Slope 0.02495 0.01437 0.084
Predictor C oef SE C oef P
Gage Linearity
S 0.0260344 R-Sq 1.6%
Linearity 0.0210960 %Linearity 2.5
A v erage 0.0028508 0.3 0.000
0.3709 0.0092222 1.1 0.000
0.374 -0.0054749 0.6 0.000
0.3764 0.0118382 1.4 0.000
0.3954 -0.0061149 0.7 0.002
0.396 -0.0127627 1.5 0.000
0.401 -0.0043691 0.5 0.000
0.4443 0.0013604 0.2 0.078
Reference Bias %Bias P
Gage Bias
Gage name: C ogniTens WLS 400M
Date of study : Nov 2011
Reported by : M. Hestness
Tolerance: ±.010
Misc: C ountersink Panel
Percent of Process Variation
Gage Linearity and Bias Study for Measured
Part-to-PartReprodRepeatGage R&R
4000
2000
0
Perc
ent
% Contribution
% Study Var
% Tolerance
Point
9
Point
8
Point
7
Poin t
6
Point
5
Point
4
Point
3
Point 21
Point 20
Poin t
2
Point 1 9
Point 18
Point 17
Point 16
Point 15
P oint
14
Point 1 3
Point 12
Point 11
Point 10
Point
1
Poin t
9
Point
8
Point
7
Point
6
Point
5
Point
4
Poin t
3
Point 2 1
Point 20
Point
2
Point 19
Point 18
P oint
17
Point 1 6
Point 15
Point 14
Point 13
Point 12
P oint
11
Point 1 0
Point
1
Point
9
Point
8
Point
7
Poin t
6
Point
5
Point
4
Point
3
Point 21
Point 20
Poin t
2
Point 1 9
Point 18
Point 17
Point 16
Point 15
P oint
14
Point 1 3
Point 12
Point 11
Point 10
Point
1
0.010
0.005
0.000
Points
Sam
ple
Range
_R=0.00350
UCL=0.00902
LCL=0
Operator 1 Operator 2 Operator 3
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
0.8
0.6
0.4
Points
Sam
ple
Mean
__X=0.5390UCL=0.5426LCL=0.5354
Operator 1 Operator 2 Operator 3
Point 9
Poi n
t 8
Point 7
Point 6
Poi nt 5
Point 4
Poi n
t 3
Poi nt 2
1
Point 2
0
Poi nt 2
Point 1
9
Point 1
8
Poi nt 1
7
Point 1
6
Poi n
t 15
Point 1
4
Point 1
3
Poi nt 1
2
Point 1
1
Poi n
t 10
Point 1
0.8
0.6
0.4
Points
Operator 3Operator 2Operator 1
0.8
0.6
0.4
Operator
Point 9
P oint
8
Point 7
Point 6
P oin t 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
P oint
1
0.8
0.6
0.4
Points
Avera
ge
Operator 1
Operator 2
Operator 3
Operator
Gage name: C ogniTens WLS 400M
Date of study : Nov 2011
Reported by : M. Hestness
Tolerance: ±.010
Misc: C ountersink Panel Point
Components of Variation
R Chart by Operator
Xbar Chart by Operator
Countersink Diameter by Points
Countersink Diameter by Operator
Points * Operator Interaction
Gage R&R (ANOVA) for Countersink Diameter
• D
I
1 m~ I 11'21&ZSZkl
I~ ~ ~I ~I lfZ)f&ZS1JD
80
Copyright 2011 Lockheed Martin
Gage R&R Study - ANOVA Method Gage R&R for Countersink Diameter
Gage name: CogniTens WLS 400M
Date of study: Nov 2011
Reported by: M. Hestness
Tolerance: ±.010
Misc: Countersink Panel Point
Two-Way ANOVA Table With Interaction Source DF SS MS F P
Points 20 3.57451 0.178725 50592.6 0.000
Operator 2 0.00003 0.000016 4.4 0.018
Points * Operator 40 0.00014 0.000004 0.7 0.859
Repeatability 126 0.00060 0.000005
Total 188 3.57528
Alpha to remove interaction term = 0.25
Two-Way ANOVA Table Without Interaction Source DF SS MS F P
Points 20 3.57451 0.178725 40100.8 0.000
Operator 2 0.00003 0.000016 3.5 0.032
Repeatability 166 0.00074 0.000004
Total 188 3.57528
Gage R&R %Contribution
Source VarComp (of VarComp)
Total Gage R&R 0.0000046 0.02
Repeatability 0.0000045 0.02
Reproducibility 0.0000002 0.00
Operator 0.0000002 0.00
Part-To-Part 0.0198579 99.98
Total Variation 0.0198625 100.00
Process tolerance = 0.02
Study Var %Study Var %Tolerance
Source StdDev (SD) (6 * SD) (%SV) (SV/Toler)
Total Gage R&R 0.002153 0.012917 1.53 64.58
Repeatability 0.002111 0.012667 1.50 63.33
Reproducibility 0.000421 0.002528 0.30 12.64
Operator 0.000421 0.002528 0.30 12.64
Part-To-Part 0.140918 0.845508 99.99 4227.54
Total Variation 0.140934 0.845606 100.00 4228.03
Number of Distinct Categories = 92
81
Copyright 2011 Lockheed Martin
ATOS Triple Scan Input Data
Operator 1 Operator 2 Operator 3
Nominal Diam. Rep 1 Rep 2 Rep 3 Rep 1 Rep 2 Rep 3 Rep 1 Rep 2 Rep 3
0.3764 0.3689 0.3706 0.3695 0.375 0.3721 0.3714 0.3748 0.3744 0.3752
0.507 0.545 0.5476 0.5477 0.545 0.5468 0.5424 0.5433 0.5438 0.5444
0.5781 0.5709 0.5706 0.5705 0.5752 0.5752 0.5733 0.575 0.5748 0.5753
0.401 0.3895 0.3898 0.3884 0.3986 0.3969 0.39 0.3999 0.3992 0.4001
0.5286 0.5227 0.5229 0.5226 0.525 0.5248 0.5222 0.5268 0.5265 0.5264
0.6403 0.6448 0.6435 0.6444 0.6461 0.6455 0.6431 0.6448 0.6453 0.6444
0.7778 0.7738 0.7824 0.7798 0.7766 0.7772 0.7741 0.7763 0.776 0.7766
0.3709 0.3617 0.3647 0.3625 0.3679 0.3693 0.3671 0.3683 0.3677 0.3685
0.4443 0.4391 0.4383 0.4397 0.4438 0.4437 0.4418 0.4433 0.4427 0.4436
0.5726 0.5692 0.5686 0.5689 0.5712 0.5721 0.5711 0.572 0.5715 0.5724
0.3954 0.3857 0.3873 0.3877 0.392 0.3914 0.3932 0.3932 0.393 0.3934
0.5199 0.5155 0.5164 0.5163 0.518 0.5173 0.5167 0.5188 0.5186 0.5185
0.6399 0.6441 0.6438 0.6438 0.6469 0.6449 0.645 0.6453 0.6449 0.6458
0.7756 0.77 0.7687 0.7708 0.7773 0.7744 0.7735 0.7741 0.7742 0.7743
0.374 0.3561 0.3629 0.3612 0.3662 0.3659 0.3635 0.3633 0.3643 0.3633
0.4495 0.4451 0.445 0.4465 0.4474 0.4478 0.4493 0.4496 0.4495 0.4498
0.5692 0.5673 0.5659 0.5663 0.5674 0.5668 0.5673 0.569 0.5696 0.5694
0.396 0.385 0.3872 0.3871 0.3942 0.3943 0.3927 0.3945 0.3947 0.3947
0.5218 0.5174 0.5175 0.5179 0.5206 0.5187 0.5187 0.5213 0.5211 0.5211
0.6464 0.6505 0.6481 0.6498 0.6512 0.6511 0.6497 0.6501 0.65 0.6507
0.7747 0.775 0.7798 0.7779 0.777 0.7762 0.7754 0.7748 0.775 0.7753
82
Copyright 2011 Lockheed Martin
0.80.70.60.50.4
0.04
0.03
0.02
0.01
0.00
-0.01
-0.02
Reference Value
Bia
s
0
Regression
95% CI
Data
Avg Bias
BiasLinearity
1.6
0.8
0.0
Pe
rce
nt
C onstant -0.008111 0.002879 0.005
Slope 0.014522 0.005214 0.006
Predictor C oef SE C oef P
Gage Linearity
S 0.0094474 R-Sq 4.0%
Linearity 0.0119675 %Linearity 1.5
A v erage -0.0003249 0.0 0.026
0.3709 -0.0044889 0.5 0.001
0.374 -0.0110333 1.3 0.000
0.3764 -0.0039667 0.5 0.001
0.3954 -0.0046333 0.6 0.001
0.396 -0.0044000 0.5 0.004
0.401 -0.0062889 0.8 0.002
0.4443 -0.0025222 0.3 0.004
Reference Bias %Bias P
Gage Bias
Gage name: A TO S Triple Scan
Date of study : Nov 2011
Reported by : M. Hestness
Tolerance: ±.010
Misc: C ountersink Panel
Percent of Process Variation
Gage Linearity and Bias Study for Measured
Part-to-PartReprodRepeatGage R&R
4000
2000
0
Perc
ent
% Contribution
% Study Var
% Tolerance
Point
9
Point
8
Point
7
Poin t
6
Point
5
Point
4
Point
3
Point 21
Point 20
Poin t
2
Point 1 9
Point 18
Point 17
Point 16
Point 15
P oint
14
Point 1 3
Point 12
Point 11
Point 10
Point
1
Poin t
9
Point
8
Point
7
Point
6
Point
5
Point
4
Poin t
3
Point 2 1
Point 20
Point
2
Point 19
Point 18
P oint
17
Point 1 6
Point 15
Point 14
Point 13
Point 12
P oint
11
Point 1 0
Point
1
Point
9
Point
8
Point
7
Poin t
6
Point
5
Point
4
Point
3
Point 21
Point 20
Poin t
2
Point 1 9
Point 18
Point 17
Point 16
Point 15
P oint
14
Point 1 3
Point 12
Point 11
Point 10
Point
1
0.008
0.004
0.000
Points
Sam
ple
Range
_R=0.001789
UCL=0.004605
LCL=0
Operator 1 Operator 2 Operator 3
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
Point 9
Point 8
Point 7
Point 6
Point 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
0.8
0.6
0.4
Points
Sam
ple
Mean
__X=0.5358UCL=0.5377LCL=0.5340
Operator 1 Operator 2 Operator 3
Point 9
Poi n
t 8
Point 7
Point 6
Poi nt 5
Point 4
Poi n
t 3
Poi nt 2
1
Point 2
0
Poi nt 2
Point 1
9
Point 1
8
Poi nt 1
7
Point 1
6
Poi n
t 15
Point 1
4
Point 1
3
Poi nt 1
2
Point 1
1
Poi n
t 10
Point 1
0.8
0.6
0.4
Points
Operator 3Operator 2Operator 1
0.8
0.6
0.4
Operator
Point 9
P oint
8
Point 7
Point 6
P oin t 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
P oint
1
0.8
0.6
0.4
Points
Avera
ge
Operator 1
Operator 2
Operator 3
Operator
Gage name: A TO S Triple Scan
Date of study : Nov 2011
Reported by : M. Hestness
Tolerance: ±.010
Misc: C ountersink Panel
Components of Variation
R Chart by Operator
Xbar Chart by Operator
Countersink Diameter by Points
Countersink Diameter by Operator
Points * Operator Interaction
Gage R&R (ANOVA) for Countersink Diameter
I I:- I
•
1 ... m~ ~~~~
~I
83
Copyright 2011 Lockheed Martin
Gage R&R Study - ANOVA Method Gage R&R for Countersink Diameter
Gage name: ATOS Triple Scan
Date of study: Nov 2011
Reported by: M. Hestness
Tolerance: ±.010
Misc: Countersink Panel
Two-Way ANOVA Table With Interaction Source DF SS MS F P
Points 20 3.39445 0.169722 16786.1 0.000
Operator 2 0.00035 0.000177 17.5 0.000
Points * Operator 40 0.00040 0.000010 5.9 0.000
Repeatability 126 0.00022 0.000002
Total 188 3.39542
Alpha to remove interaction term = 0.25
Gage R&R %Contribution
Source VarComp (of VarComp)
Total Gage R&R 0.0000072 0.04
Repeatability 0.0000017 0.01
Reproducibility 0.0000054 0.03
Operator 0.0000027 0.01
Operator*Points 0.0000028 0.01
Part-To-Part 0.0188569 99.96
Total Variation 0.0188641 100.00
Process tolerance = 0.02
Study Var %Study Var %Tolerance
Source StdDev (SD) (6 * SD) (%SV) (SV/Toler)
Total Gage R&R 0.002678 0.016066 1.95 80.33
Repeatability 0.001313 0.007876 0.96 39.38
Reproducibility 0.002334 0.014003 1.70 70.01
Operator 0.001628 0.009768 1.19 48.84
Operator*Points 0.001672 0.010033 1.22 50.16
Part-To-Part 0.137320 0.823923 99.98 4119.61
Total Variation 0.137347 0.824079 100.00 4120.40
Number of Distinct Categories = 72
84
Copyright 2011 Lockheed Martin
Rexcan 4 White Light Scanner Input Data
Operator 1 Operator 2 Operator 3
Nominal Diam. Rep 1 Rep 2 Rep 3 Rep 1 Rep 2 Rep 3 Rep 1 Rep 2 Rep 3
0.3764 0.3838 0.3726 0.3708 0.3808 0.3803 0.3803 0.3685 0.3829 0.373
0.507 0.5417 0.5401 0.5379 0.543 0.5446 0.5442 0.5497 0.5366 0.5426
0.5781 0.5715 0.569 0.572 0.5785 0.5753 0.5824 0.5733 0.5758 0.5746
0.401 0.4026 0.3979 0.4008 0.4018 0.4058 0.4075 0.4032 0.402 0.4041
0.5286 0.5342 0.5226 0.5285 0.5262 0.5343 0.5323 0.5274 0.5273 0.5256
0.6403 0.6389 0.6384 0.6365 0.6453 0.6473 0.6438 0.6334 0.6433 0.639
0.7778 0.7767 0.771 0.7772 0.7746 0.7864 0.779 0.7731 0.7756 0.7809
0.3709 0.3677 0.3656 0.3664 0.3755 0.3842 0.3779 0.3601 0.3738 0.3605
0.4443 0.4415 0.4387 0.4329 0.4547 0.4522 0.4662 0.4355 0.4493 0.4361
0.5726 0.57 0.5654 0.5661 0.5731 0.5787 0.5776 0.566 0.5741 0.571
0.3954 0.3923 0.391 0.3904 0.3946 0.3985 0.4002 0.3894 0.3944 0.3883
0.5199 0.5173 0.5158 0.515 0.5224 0.5229 0.5281 0.5157 0.5199 0.5143
0.6399 0.6409 0.6402 0.6384 0.6468 0.6522 0.653 0.6425 0.6492 0.6427
0.7756 0.7716 0.7683 0.7648 0.7761 0.7775 0.7832 0.7704 0.7767 0.7692
0.374 0.3595 0.3604 0.3662 0.368 0.3766 0.3744 0.3509 0.3646 0.356
0.4495 0.4573 0.4578 0.4557 0.4582 0.4648 0.4582 0.4444 0.4556 0.4449
0.5692 0.5675 0.573 0.5754 0.577 0.5761 0.5755 0.5653 0.5742 0.5621
0.396 0.3902 0.392 0.3975 0.3955 0.4006 0.4022 0.3893 0.3993 0.3875
0.5218 0.518 0.5229 0.5234 0.5236 0.5237 0.5274 0.5207 0.5272 0.5123
0.6464 0.6468 0.6503 0.6479 0.6499 0.6557 0.651 0.6456 0.651 0.6465
0.7747 0.7752 0.7777 0.7719 0.7754 0.78 0.7781 0.7722 0.7788 0.7756
85
Copyright 2011 Lockheed Martin
0.80.70.60.50.4
0.05
0.04
0.03
0.02
0.01
0.00
-0.01
-0.02
Reference Value
Bia
s
0
Regression
95% CI
Data
Avg Bias
BiasLinearity
0.2
0.1
0.0
Pe
rce
nt
C onstant 0.000738 0.002970 0.804
Slope 0.001714 0.005379 0.750
Predictor C oef SE C oef P
Gage Linearity
S 0.0097462 R-Sq 0.1%
Linearity 0.0013954 %Linearity 0.2
A v erage 0.0016571 0.2 0.000
0.3709 -0.0007111 0.1 0.793
0.374 -0.0099333 1.2 0.009
0.3764 0.0006000 0.1 0.728
0.3954 -0.0021667 0.3 0.137
0.396 -0.0011000 0.1 0.512
0.401 0.0018556 0.2 0.117
0.4443 0.0009333 0.1 0.803
Reference Bias %Bias P
Gage Bias
Gage name: Rexcan 4 White Light Sy stem
Date of study : Nov 2011
Reported by : M. Hestness
Tolerance: ±.010
Misc: C ountersink Panel
Percent of Process Variation
Gage Linearity and Bias Study for Measured
Part-to-PartReprodRepeatGage R&R
4000
2000
0
Perc
ent
% Contribution
% Study Var
% Tolerance
Point
9
Point
8
Point
7
Point
6
Poin t
5
Point
4
Point
3
Point 21
Point 20
Point
2
Point 19
Point 18
P oin t
17
Point 1 6
Point 15
Point 14
Point 13
Point 12
Point 11
Point 10
Point
1
Point
9
Point
8
Point
7
Point
6
Point
5
P oint
4
Poin t
3
Point 2 1
Point 20
Point
2
Point 19
Point 18
Point 17
Point 16
P oin t
15
Point 1 4
Point 13
Point 12
Point 11
Point 10
Point
1
Poin t
9
Point
8
Point
7
Point
6
Point
5
Point
4
Point
3
Point 21
P oin t
20
Point
2
Point 19
Point 18
Point 17
Point 16
Point 15
Point 14
P oin t
1 3
Point 12
Point 11
Point 10
Point
1
0.02
0.01
0.00
Points
Sam
ple
Range
_R=0.00682
UCL=0.01756
LCL=0
Operator 1 Operator 2 Operator 3
Point 9
Point 8
Point 7
Point 6
P oin t
5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Po in
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Po in
t 10
Point 1
Point 9
Point 8
Point 7
Point 6
Point 5
P oint
4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Po in
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
P oin t
9
Point 8
Point 7
Point 6
Point 5
Point 4
Point 3
Po in
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Po in
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
Point 1
0.8
0.6
0.4
Points
Sam
ple
Mean
__X=0.5378UCL=0.5448LCL=0.5308
Operator 1 Operator 2 Operator 3
Point 9
Poi n
t 8
Point 7
Point 6
Poi nt 5
Point 4
Poi n
t 3
Poi nt 2
1
Point 2
0
Poi nt 2
Point 1
9
Point 1
8
Poi nt 1
7
Point 1
6
Poi n
t 15
Point 1
4
Point 1
3
Poi nt 1
2
Point 1
1
Poi n
t 10
Point 1
0.8
0.6
0.4
Points
Operator 3Operator 2Operator 1
0.8
0.6
0.4
Operator
Point 9
P oint
8
Point 7
Point 6
P oin t 5
Point 4
Point 3
Poin
t 21
Poin
t 20
Point 2
Poin
t 19
Poin
t 18
Poin
t 17
Poin
t 16
Poin
t 15
Poin
t 14
Poin
t 13
Poin
t 12
Poin
t 11
Poin
t 10
P oint
1
0.8
0.6
0.4
Points
Avera
ge
Operator 1
Operator 2
Operator 3
Operator
Gage name: Rexcan 4 White Light Scanner
Date of study : Nov 2011
Reported by : M. Hestness
Tolerance: ±.010
Misc: C ountersink Panel
Components of Variation
R Chart by Operator
Xbar Chart by Operator
Countersink Diameter by Points
Countersink Diameter by Operator
Points * Operator Interaction
Gage R&R (ANOVA) for Countersink Diameter
D
lj~ I
I$ $ $1 ~EJ
86
Copyright 2011 Lockheed Martin
Gage R&R Study - ANOVA Method Gage R&R for Countersink Diameter
Gage name: Rexcan 4 White Light Scanner
Date of study: Nov 2011
Reported by: M. Hestness
Tolerance: ±.010
Misc: Countersink Panel
Two-Way ANOVA Table With Interaction Source DF SS MS F P
Points 20 3.30650 0.165325 6759.36 0.000
Operator 2 0.00212 0.001061 43.40 0.000
Points * Operator 40 0.00098 0.000024 1.45 0.063
Repeatability 126 0.00213 0.000017
Total 188 3.31172
Alpha to remove interaction term = 0.25
Gage R&R %Contribution
Source VarComp (of VarComp)
Total Gage R&R 0.0000359 0.19
Repeatability 0.0000169 0.09
Reproducibility 0.0000190 0.10
Operator 0.0000165 0.09
Operator*Points 0.0000025 0.01
Part-To-Part 0.0183667 99.81
Total Variation 0.0184026 100.00
Process tolerance = 0.02
Study Var %Study Var %Tolerance
Source StdDev (SD) (6 * SD) (%SV) (SV/Toler)
Total Gage R&R 0.005988 0.035929 4.41 179.64
Repeatability 0.004107 0.024641 3.03 123.21
Reproducibility 0.004358 0.026148 3.21 130.74
Operator 0.004057 0.024343 2.99 121.72
Operator*Points 0.001591 0.009545 1.17 47.73
Part-To-Part 0.135524 0.813143 99.90 4065.71
Total Variation 0.135656 0.813936 100.00 4069.68
Number of Distinct Categories = 31
Top Related