Report on SIM.L-K1 (SIM.4.2) Regional Comparison
Stage One: Calibration of Gauge Blocks by Optical Interferometry
Final Report — 27 September 2006
J. E. Decker1, J. Altschuler, H. Beladie, I. Malinovsky, E. Prieto, J. Stoup, A. Titov, M. Viliesid, J. R. Pekelsky1
1Dimensional Metrology Program Institute for National Measurement Standards (INMS)
National Research Council Canada (NRC) Ottawa, CANADA K1A 0R6
Abstract
Results of the Stage One portion of the Inter-American System of Metrology (SIM) regional international comparison of gauge block calibration by optical interferometry are presented. In this measurement roundrobin, short gauge blocks, 6 made of steel and 6 made of tungsten carbide, in the range of nominal length from 2 mm to 100 mm, were calibrated by 5 national metrology institutes (NMIs) of the SIM region, and one NMI from EUROMET. By employing the technique of optical interferometry, each of the laboratories establishes a direct link to their national primary standard of length through the calibrated laser wavelengths. Results of central length calibration are presented and discussed with regard to vacuum wavelength correction for refractive index of air, phase-change on reflection and wringing effects. Measurement uncertainty evaluation is also discussed.
1send correspondence to JED 207 M-36 Montreal Road, NRC, Ottawa, CANADA, K1A 0R6, tel. (613)991-1633, fax. (613)9521394, e-mail: jennifer.decker@nrc.ca
1
Contents
1 Introduction
3
2 Participants
3
3 Gauge Block Artefacts
5
4 Calibration Technique
5
5 Results and Discussion
7
5.1 Central Length Measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
5.2 Difference Between Left and Right Measurement Face Wringing . . . . . . . . . . . . . . . . . . 13
5.3 Phase Correction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
5.4 Measurement Uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
6 Conclusions
16
7 Acknowledgements
19
A Evaluation of the Comparison Reference Value
21
A.1 Evaluations and Calculations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
A.2 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
B Degrees of Equivalence
39
C Linking of SIM.4.2 Gauge Block Comparison to the CCL-K1 Gauge Block Comparison 40
D Tables of Bilateral Equivalence
40
2
1 Introduction
There are several goals associated with an international comparison of measuring artefacts. One goal is to probe the current level of world capability, which then forms the basis of consensus agreement on current state-of-the-art. This international comparison is in the context of absolute central length measurement of gauge blocks.
A benefit to participating laboratories is that each one is able to test its performance. Each participating laboratory has the goal of verifying that their overall measurement system is functioning correctly. This is meaningful because gauge block calibration by optical interferometry involves many sophisticated techniques to form the relationship between the vacuum wavelength of laser-light [1] and the overall mechanical length of a gauge block. These calibrations are a fundamental first step in the chain of traceability to the definition of the metre. Corrections for physical nature of a gauge measuring surface can be elusive placed in the context of wavelengths of light, which are accepted as representing our primary scale with which to establish traceability. International comparison offers the only method to scientifically observe and interrogate the biases that exist in these measurements, even though all the labs are using the same technique, and in some cases even the same instrumentation, and yet also claim direct traceability to the definition of the metre and the ITS-90 temperature scale. And finally, one of the most recent and important goals of international comparison of measurement capability is to support the international Mutual Recognition Arrangement (MRA) [2].
This paper provides a detailed report of the Stage One of a multi-stage international comparison which samples the gauge block calibration service offered to clients by national metrology institutes (NMIs) comprising the SIM region. Stage One comprises the gauge block calibrations by the technique of optical interferometry during the time period of June 1998 to December 1999. The same gauge blocks are circulated in Stage Two where SIM NMIs calibrate these gauge blocks using mechanical comparison technique. The same gauge blocks are used successive stages of the comparison, therefore results can shed insight on the link between interferometric and mechanical comparison techniques in addition to all NMIs participating in both stages. A second report outlines the results obtained by mechanical comparison [3]. Several laboratories, namely NIST (USA), INMETRO (Brazil), INTI (Argentina), and CEM (Spain) calibrated the gauge blocks using both techniques thus providing a solid link between the two comparisons.
2 Participants
Out of the six national metrology institutes (NMIs) participating in this comparison, five represent countries of the SIM region and one country represented the EUROMET region. The laboratories, their representative acronyms, and contact information are listed in Table 1. There were many challenges associated with transport and customs issues, however circulation of the gauge blocks to all the labs took about 18 months. The tour circuit for Stage One is outlined in Table 2. INTI, NIST and CEM calibrated the gauge blocks by both interferometry and mechanical comparison techniques, so the time line for these labs is somewhat extended.
The gauge blocks were measured at INMETRO by two completely different gauge block interferometer instruments, also involving different staff members. Data denoted by INMETRO1 represent results from a researchgrade instrument for which client calibrations are offered on request. Results denoted by INMETRO2 represent results from the routine interferometric gauge block calibration service offered by INMETRO as listed in the Key Comparison Database (KCDB) Appendix C: Calibration and Measurement Capabilities.
3
Laboratory Contact Information
Phone, E-mail
CENAM INMETRO1 INMETRO2 INTI
NIST CEM NRC (pilot)
Miguel Viliesid Alonso Metrologia Dimensional Centro Nacional de Metrolog´ıa (CENAM) Apartado Postal 1-100 Centro 76000 Queretaro Queretaro, Mexico C. A. Massone, I. Malinovsky Instituto Nacional de Metrolgia, Normalizac¸˜ao
e Qualidade Industrial (INMETRO) Av. N. S. das Graas 50 Duque de Caxias, Rio de Janeiro, Brazil Hakima Beladie Instituto Nacional de Metrolgia, Normalizac¸˜ao
e Qualidade Industrial (INMETRO) Av. N. S. das Graas 50 Duque de Caxias, Rio de Janeiro, Brazil Jeronimo Altschuler Centro de Investigaci´on y Desarrollo en Fisica Instituto Nacional de Tecnologia Industrial (INTI) Parque Tecnolgico Miguelete: Av. General Paz entre Albarellos y Constituyentes CC 157 - (1650) San Martin Buenos Aires, Argentina John Stoup National Institute of Standards and Technology (NIST) Room B113, Metrology Building Gaithersburg, MD 20899-0001 USA Emilio Prieto Esteban Centro Espanol de Metrologia (CEM) Alfar, 2 - 28760 Tres Cantos Madrid Spain Jennifer Decker Institute for National Measurement Standards (INMS) National Research Council Canada (NRC) Ottawa, K1A 0R6, Canada
Tel. +52 42 11 0574 Fax +52 42 11 0577 e-mail: mviliesi@cenam.mx
Tel. +55 21 502-1009 Fax. +55 21 293-6559 e-mail: laint@inmetro.gov.br
Tel. +55 21 502-1009 Fax. +55 21 293-6559 e-mail: laint@inmetro.gov.br
Tel. +54 11 4752-5402 Fax +54 11 4713-4140 e-mail: jeroa@inti.gov.ar
Tel. +1 301 975 3476 Fax +1 301 869 0822 e-mail: John.Stoup@nist.gov
Tel. +34 91 8074 716 / 700 Fax +34 91 8074 807 e-mail: eprieto@cem.es
Tel. +1 613 991 1633 Fax +1 613 952 1394 e-mail: Jennifer.Decker@nrc.ca
Table 1: Participants of SIM.4.2 regional comparison of gauge block calibration, Stage One by optical interferometry.
4
3 Gauge Block Artefacts
A total of 12 rectangular gauge blocks, ISO 3650 [6] Grade K, were selected. The nominal lengths of the gauge blocks were chosen to provide adequate representation to the range and sampling of short gauge blocks. Moreover, the gauge block nominal lengths echo the short gauge blocks used in the CCL-K1 key comparison. Gauge block materials of steel (CARY, Switzerland) and tungsten carbide (Select, UK) were employed. The nominal lengths of the steel gauge blocks are: 2 mm, 5 mm, 8 mm, 10 mm, 50 mm, 100 mm, and for the tungsten carbide gauge blocks: 2 mm, 5 mm, 8 mm, 20 mm, 50 mm, 100 mm. Gauge blocks were housed in a wooden case. Every attempt was made to hand-carry the gauge blocks whenever possible (July 1998), however, from thereafter were shipped from one laboratory to another because of limited resources and limited travelers between NMIs. Often times shipping took a time duration of one month.
Thermal expansion coefficients for the gauge blocks were not measured, rather the values provided by the manufacturer were requested to be used. These values are: 11.5 × 10−6 /K for steel, and 5.0 × 10−6 /K for tungsten carbide. INMETRO2 used a value of 4.23 × 10−6 /K for tungsten carbide. INMETRO1 measured 4.25×10−6 /K for the thermal expansion coefficient of the 100 mm and 50 mm gauge blocks with an uncertainty of 5.0 × 10−8 /K.
Gauge blocks were inspected for damage immediately upon arrival at each laboratory, and a detailed report form outlining the integrity of each gauge block was faxed back to the pilot lab upon receipt of the gauge blocks. Following the first stage of the comparison the gauge blocks were in sufficiently good condition to consider continuing with these gauge blocks for Stage Two of the comparison. Some of the gauge blocks had small scratches, but most of the measuring faces demonstrated good to fair wringing properties following Stage One. The right side of the 50 mm steel gauge block was reported damaged at INMETRO, and was not wringable by INTI or NIST. Indeed, when the pilot lab measured the gauge blocks in October 1999 the right face of the 50 mm steel gauge block was found to be slightly damaged and wringing was compromised. The left face of the 100 mm steel gauge block was difficult to wring following Stage One (it was not wringable by NRC), although INTI, NIST and CEM managed to wring and measure that face.
The gauge blocks appear to be stable in length during the time of the comparison. The pilot lab measured the gauge block at regular intervals in an attempt to monitor the stability of the gauge blocks. During Stage One of this 2-stage comparison, NRC measured the gauge blocks twice. Because of logistical challenges, the pilot lab measured the gauge blocks at a frequency of not more than once per year.
4 Calibration Technique
All participants calibrated the gauge blocks by the technique of optical interferometry, applying the method of exact fractions. Table 3 summarizes details of the equipment used by each laboratory. Most laboratories establish traceability to the definition of the metre through calibration of laser frequency against an iodinestabilized He-Ne primary standard laser in-house. INTI obtains their traceability through calibration of laser vacuum wavelength by National Physical Laboratory (NPL, UK). Some participating laboratories use lamps as light sources, in particular the cadmium-114 isotope lamp. The electrodeless cadmium-114 lamp correctly operated is considered to be a length standard by the CIPM [1, 12], with absolute accuracy of 7 × 10−8 (k = 3 relative uncertainty) in vacuum wavelength, and therefore does not require further calibration to establish traceability to the definition.
The protocol document specified that the gauge blocks are to be calibrated in accordance with the standard
5
Laboratory Dates of Measurement Results Received
NRC (pilot) CENAM INMETRO2 INMETRO1 INTI NIST NRC (pilot) CEM
June 1998 August, September 1998
November 1998 – February 1999 March, April 1999 June, July 1999
October 1999 December 1999 – January 2000
— 28 January 1999 11 March 1999 25 February 1999
11 June 1999 13 January 2000
— 28 April 2000
Table 2: Tour time-line of SIM.4.2 regional comparison of gauge block calibration, Stage One by optical interferometry.
Laboratory
Instrument & Light Sources
Fringe Evaluation
NRC CENAM INMETRO1 INMETRO2 INTI NIST CEM
NRC Twyman-Green [4, 5] He-Ne lasers 633, 543, 612 nm Twyman-Green NPL-TESA
He-Ne lasers 633, 543 nm Carl Zeiss (modified) He-Ne laser 633 nm Jena-Zeiss
114-Cd lamp 644, 509, 480, 468 nm Twyman-Green NPL-TESA He-Ne lasers 633, 543 nm
Fizeau NPL Hilger-Watts (modified) He-Ne laser 633 nm
Twyman-Green NPL-TESA He-Ne lasers 633, 543 nm
Localisation by eye to fiducial in video image automated DIP of fringe pattern custom DIP of fringe pattern visual interpolation automated DIP of fringe pattern visual interpolation automated DIP of fringe pattern
Table 3: Summary of instruments and light sources used in gauge block calibrations of the SIM.4.2 Regional Comparison. DIP: Digital Image Processing
6
ISO 3650 [6], namely that central gauge block length is defined as the height of the centre point of the gauge block measuring face with respect to an auxiliary plane surface. One measuring face of the gauge block is wrung to the auxiliary surface and measured. The gauge block is turned end-over-end and the other measuring face is wrung to the platen and likewise measured. This sequence is repeated so that the result reported by each participant is an average of four separate wringing measurements. In keeping with the ISO 3650 guidelines, certain specific information was requested to be reported by each participant. These data are discussed in turn below.
Following convention of reporting gauge block central length, l is reported as the average of the left and right measuring face wringings, as a deviation d from nominal length L,
d=l−L
(1)
where a plus sign indicates that the gauge block is longer than the nominal length, and a minus sign that it is shorter.
In gauge block interferometry the largest correction for environmental influences is the adjustment of the vacuum wavelength of light for the refractive index of air λv = nλair. All laboratories applied measured values of air temperature, pressure and partial pressure of water vapour to empirical formulae modeling the behaviour of the refractive index of air. Labs varied in the version of the Edl´en equation in use. NRC and INTI applied the Birch and Downs 1994 [8] version, CENAM and NIST refer to an update made in 1998 [9]. The other participants did not specify which version was applied. No refractometers were used in this comparison.
5 Results and Discussion
5.1 Central Length Measurement
Laboratories submitted detailed reports including: average deviation from nominal central length d, average deviation from nominal length for right and left face wringings, platen materials and phase corrections, standard uncertainty components, combined standard uncertainty and degrees of freedom. Central length measurement values and standard uncertainties reported by each participant are tabulated in Tables 4 through 7, and plotted in Figures 1 and 2 for all gauge blocks of the comparison2.
The simple arithmetic mean
1n
x = n xi
(2)
x=1
is included in the data plots, where xi is the central length measurement reported by each laboratory and n is the number of participants. The NRC pilot measurements intended to probe gauge block stability are included in the plots for information, since there is not enough of this pilot data to warrant a separate plot for Stage One. Only the first measurement of NRC is used in the evaluation of the mean and the KCRV. The exclusive arithmetic mean [10] is also shown in the plots. The exclusive mean is evaluated by taking the mean of all laboratories, leaving out the result of the participant laboratory. This technique allows graphical demonstration of the amount of correlation of each participant with the ‘world’ simple arithmetic mean.
2An oversight in the submission of CENAM was revealed during data tabulation. Measurement results and uncertainties used in the original computations and Draft A Report were confirmed to be correct, and the final Draft B Report remains unchanged from the one accepted by the CCL-WGDM in Sept 2005.
7
8
Figure 1: Plot of central length expressed as deviation from nominal length reported by each participant for steel gauge blocks. Thick error bars represent the standard uncertainty, while longer thin error bars represent k95u(xi) where k95 = tp(νi) from the Student’s t-distribution for standard uncertainties u(xi) and degrees of freedom νi submitted by the participants. The solid line represents the simple arithmetic mean of the reported central lengths. The dash for each participant represents the exclusive simple arithmetic mean (see text).
9
Nominal Length /mm
2 5 8 10 50 100
Deviation from Nominal Length for Steel Gauge Blocks /nm
NRC
18 -52 29 35 31 -124
CENAM
37 -51 45 22 36 -93
INMETRO2
60 -31 81 51 58 -68
INMETRO1
36 -58 42 19 30 -98
INTI
33 -65 40 14 19 -104
NIST
42 -52 59 8 36 -100
CEM
26 -65 47 -4 9 -148
Table 4: Central length expressed as deviation from nominal length reported by each participant for steel gauge blocks.
Nominal Length /mm
2 5 8 10 50 100
NRC
14 14 14 14 18 26
Standard Uncertainties for Steel Gauge Blocks /nm
CENAM
7 7 7 7 13 23
INMETRO2
14 14 14 14 19 29
INMETRO1
2 2 2 2 3 4
INTI
11 11 11 11 14 21
NIST
9 9 10 10 13 18
CEM
8 8 8 9 11 17
Table 5: Combined standard uncertainty attributed to steel gauge block central length measurement as reported by each participant.
Nominal Length /mm
2 5 8 20 50 100
Deviation from Nominal Length for Tungsten Carbide Gauge Blocks /nm
NRC
-10 19 45 10 -25 -58
CENAM
0 35 54 21 -13 -27
INMETRO2
-8 16 34 1 -24 -28
INMETRO1
-18 10 30 -5 -40 -36
INTI
-20 10 28 -2 -36 -57
NIST
-23 19 31 18 -32 -46
CEM
0 13 39 17 -43 -63
Table 6: Central length expressed as deviation from nominal length reported by each participant for tungsten carbide gauge blocks.
10
Nominal Length /mm
2 5 8 20 50 100
Standard Uncertainties for Tungsten Carbide Gauge Blocks /nm
NRC
14 14 14 14 17 24
CENAM
14 14 14 15 22 36
INMETRO2
14 14 14 15 18 26
INMETRO1
2 1 1 2 3 4
INTI
11 11 11 11 12 16
NIST
9 9 10 11 13 18
CEM
8 8 8 9 10 13
Table 7: Combined standard uncertainty attributed to tungsten carbide gauge block central length measurement reported by each participant.
11
12
Steel Tungsten Carbide
maximum st dev
maximum st dev
Difference Between Left and Right Measuring Face Wringing /nm NRC CENAM INMETRO2 INMETRO1 INTI NIST CEM
17
9
20
9
6
11
12
2
15
17
7
1
9
9
16
15
31
10
8
11
13
12 16 17
6
7
8
6
Table 8: Maximum difference between central length measurements of left and right wringings listed with the standard deviation of this difference for all gauge blocks in each material sample (nm units).
5.2 Difference Between Left and Right Measurement Face Wringing
The protocol requested that participants report the average d for right and left measuring face wringings. Figures 3 and 4 plot these results for steel and tungsten carbide gauge blocks respectively. Differences in length measurements between left and right side wringings can indicate a geometry feature of the gauge block that results in different wringing qualities between left and right. The quality of the gauge block, the platen and the technical experience of the metrologist all influence the the closeness of left and right wring length measurements. Participants of this comparison showed similar results in this category.
5.3 Phase Correction
According to the ISO 3650 definition of gauge block length, the central length measurement must include the appropriate corrections for difference in material or surface texture between the platen and the gauge block measuring face [6, 7]. INMETRO1 used their technique of reproducible wringing [11], whereas all other participants used stack techniques for the evaluation of their correction for phase-change on reflection. INMETRO2 did not perform pack experiments on the steel gauge blocks as they did on the tungsten carbide gauge blocks, but rather applied a phase correction for steel based on previous characterization experiments. The comparison protocol included specific instructions for reporting these phase change correction values. Submitted values are listed in Table 9.
5.4 Measurement Uncertainty
To expedite analysis of comparison comparison results, labs were requested to provide a measurement uncertainty budget in the model of that described in [13], including the standard uncertainty components attributed to the largest influence parameters of their calibration. An example table was provided in the protocol document.
Each laboratory submitted a summary evaluation of their measurement uncertainty. The following influence parameters were identified in the submitted uncertainty evaluations:
• λi: vacuum wavelength of the light sources, • Fi: measurement of interference fringe fraction,
13
Figure 2: Plot of central length expressed as deviation from nominal length reported by each participant for tungsten carbide gauge blocks. Thick error bars represent the standard uncertainty, while longer thin error bars represent k95u(xi) where k95 = tp(νi) from the Student’s t-distribution for standard uncertainties u(xi) and degrees of freedom νi submitted by the participants. The solid line represents the simple arithmetic mean of the reported central lengths. The dash for each participant represents the exclusive simple arithmetic mean (see Section 5.1 text).
Figure 3: Plot of differences between left and right wringing compared to average central length for steel gauge blocks.
14
Figure 4: Plot of differences between left and right wringing compared to average central length for each participant, for tungsten carbide gauge blocks.
Laboratory
NRC CENAM INMETRO1 INMETRO2 INTI NIST CEM
Steel Gauge Blocks
Platen
Phase
Material
Correction /nm
fused silica steel (TESA, UK) steel (Cary, CH)
quartz steel (TESA, UK)
+51 −23 ‘slave block’ +45 −46
steel
+11
steel (TESA, UK)
−17
Tungsten Carbide Gauge Blocks
Platen
Phase
Material
Correction /nm
fused silica tungsten carbide (TESA, UK)
steel quartz tungsten carbide (TESA, UK) steel tungsten carbide (TESA, UK)
+43 −20 ‘slave block’ +19 −39 −11.7 +13
Table 9: Summary of platen materials and phase corrections for steel and tungsten carbide gauge blocks.
15
• n: refractive index of air (combined standard uncertainty includes components of air temperature, air pressure and relative humidity measurements),
• Δtg: gauge block temperature measurement, • α: linear coefficient of thermal expansion, • δlΩ: obliquity correction – alignment of the entrance aperture, • Δls: obliquity correction – size of the source aperture, • δlA: wavefront aberrations, • δlG: departure from perfect prismatic geometry of the gauge block, • δlw: wringing, • Δlφ: phase correction (combined standard uncertainty).
Combined standard uncertainty values reported for each gauge block in the comparison are listed in Tables 3 and 5 for the steel and tungsten carbide gauge blocks respectively. Table 10 provides a general summary of the range of expanded uncertainty and degrees of freedom for the steel gauge blocks, for nominal gauge block lengths 2 mm to 100 mm. Individual components of standard uncertainty reported by each participant are listed in Table 11.
Two of the largest influences on gauge block calibration by optical interferometry are air pressure and temperature which in turn, directly influence refractive index of air n. Uncertainty components for these influences are nested in the total uncertainty for refractive index. Temperature also affects uncertainties related to the thermal expansion of the gauge block through Δtg.
The histogram of the pooled comparison data shown in Figure 5 demonstrate that comparison results follow a distribution similar to the normal distribution. More importantly, examination of the data plots and the histograms demonstrate that there are no obvious outliers in the comparison data. Outlier data points could affect the evaluation of the reference value in a detrimental way by falsely pulling the mean in the direction of the outlier data point. Therefore all data submitted by the participants can be used in the evaluation of the reference values.
6 Conclusions
Results of SIM.4.2 regional comparison of gauge block calibration by optical interferometry are reported. Data are presented in the form of tables and plots of deviation from nominal length reported by each participating laboratory for each gauge block of the comparison. Comparison data for left and right wringing differences, uncertainty evaluations, and equipment styles are also reported. Gauge block nominal lengths and materials were selected to probe the range of nominal lengths between 2 mm and 100 mm.
The simple arithmetic mean of the central length measurement is evaluated for each gauge block in the comparison, and is recommended as the comparison reference value (KCRV). Tables in the Appendix list the difference between the individual result of each participant with respect to the KCRV, along with the expanded uncertainty of this difference. Tables of bilateral equivalence are also included in the Appendix.
16
Laboratory
NRC CENAM INMETRO1 INMETRO2 INTI NIST CEM
Maximum Temperature
Variation During
Measurements /◦C
±0.03 ±0.25 ±0.15 ±0.25 ±0.1 +0.2 ±0.05
Range of Expanded Uncertainty (Steel) /nm
28 – 52 14 – 46
3–9 28 – 58 22 – 42 18 – 36 16 – 34
Range of Degrees of Freedom (Steel)
10 – 73 72 – 251
2 – 21 13 – 256 75 – 1086
— 71 –261
Table 10: Summary of details regarding temperature range during measurements, reported expanded uncertainties and degrees of freedom for range ‘boundary’ values of 2 mm and 100 mm nominal (steel) gauge block lengths.
Components of Standard Uncertainty /nm NRC CENAM INMETRO1 INMETRO2 INTI NIST CEM
λi
0.5
3
0.2
Fi
2
3
0.1
n
20
2.5
3.4
Δtg 7.2
16.7
1.7
α
3
15
0.75
δlΩ 0.8
0.6
0.3
Δls 0.2
0.4
—
δlA 3
3.5
0.3
δlG 3
1.4
0.1
δlw
8
3
1.5
Δlφ 10
4
0.5
uc
25
24
4
1.5
3 0.3 1.1
6.6
3 4.5 4.2
15.3
8.9 3 7.5
12.9
13.8 8.5 10
15
5.7 0.8 3.5
—
0.6 0.1 0.6
0.5
0.3 — 0.2
3
3.4 3
3
2
1.4 — 2.5
6
7
4
3
10
6
5.8
5
29
21 13 15
Table 11: Summary of components of standard uncertainty for gauge block calibration by interferometry reported by participants of the SIM.4.2 Regional Comparison. Length dependent terms are in italics and are based on 100 mm nominal gauge block length.
17
Figure 5: Histogram combining all gauge blocks of the comparison. Number of occurrences of difference value from the simple arithmetic mean.
Participating laboratories demonstrate general agreement in measurement of central gauge block length within the average scatter of data around ±25 nm. Comparison data is tighter for the tungsten carbide gauge block samples, likely owing to the more reproducible wringing quality of these blocks. The surface characteristics of the tungsten carbide gauge blocks are better for wringing than steel, and the material is more durable in a comparison exercise.
Dispersion observed in the values of phase correction reported by participants using the same platens from the same manufacturers invite further investigation into the variation of phase correction values even while employing the same equipment.
Stage One of this comparison took a total of 18 months to complete. The gauge blocks returned in reasonable condition, deemed sufficiently good to continue to Stage Two of the SIM.4.2 regional gauge block comparison [3]. In Stage Two, participants calibrate the same gauge blocks by mechanical comparison methods.
Three of the seven gauge block interferometer instruments were identical instruments purchased from the same manufacturer. Agreement is demonstrated between laboratories with automated fringe evaluation, particularly for gauge block measurements of shorter nominal length.
In future comparisons, it would be advantageous to report detailed measurement uncertainty of air temperature and pressure measurement rather than overall refractive index. The influence of air pressure and temperature on vacuum wavelength corrections could have correlation with comparison results.
This comparison provides a link to the CCL-K1 Key Comparison of short gauge block calibration by interferometry through NIST, NRC and CENAM. This Comparison also provides a link to the EUROMET Comparison of short gauge block calibration through CEM, and to the SIM.4.2 Stage Two Short Gauge Block Comparison by Mechanical Comparison through NIST, INTI, CEM, and INMETRO.
18
7 Acknowledgements
The SIM.4.2 Regional Gauge Block Comparison participants gratefully acknowledge the financial support of the SIM regional committee. NRC acknowledges the technical support of M. Dagenais for gauge block calibration work, and L. Munro for maintenance of the NRC Gauge Block Interferometer Laboratory. CENAM would like to acknowledge Juan Carlos Zarraga and Carlos Colin for performing the careful measurements for this comparison. INTI would like to acknowledge Jorge Alvarez and Sergio Ilieff for having taken part in the measurements. CEM would like to acknowledge Joaqu´ın Rodriguez for performing both interferometric and mechanical comparison measurements for this comparison. The author also thanks NRC-INMS colleagues Alan Steele, Rob Douglas and Barry Wood for fruitful discussions.
References
[1] Quinn, T.J., 2003, “Practical realization of the definition of the metre, including recommended radiations of other optical frequency standards (2001),” Metrologia, 40, 103–133.
[2] http://www.bipm.fr
[3] Decker, J.E., Alschuler J., Castillo Candanedo, J., De la cruz, E., Prieto Esteban, E., Morales, R., Valente de Oliveira, J.C., Stone, J., Stoup, J., Pekelsky, J.R., 2003, “SIM.4.2 Regional Comparison Stage Two: Calibration of gauge blocks by mechanical comparison,” Metrologia, 40 No. 1A, 04003.
[4] Decker, J.E. and Pekelsky, J.R., “Gauge Block Calibration by Optical Interferometry at the National Research Council Canada,” presented at the Measurement Science Conference, Pasadena, California, 23–24 January 1997, NRC Document No. 40002.
[5] Decker J.E., Bustraan, K., de Bonth, S., Pekelsky, J.R., “Updates to the NRC gauge block interferometer,” NRC Document No. 42753, August 2000, 9 pages.
[6] ISO 3650:1998(E), 1998, 2nd Edition 1998-12-15, (International Organization for Standardization (ISO) Central Secretariat, Switzerland, Internet iso@iso.ch).
[7] Leach, R.K., Hart, A., 2000, “EUROMET Project 413: Interlaboratory comparison of measurements of the phase correction in the field of gauge block interferometry,” Metrologia, 37, 261–267.
[8] Birch, K.P. and Downs, M.J., “Correction to the Updated Edl´en Equation for the Refractive Index of Air,” Metrologia, 31, 315–316 (1994).
[9] Bo¨nsch, G., Potulski, E., 1998, “Measurement of the refractive index of air and comparison with modified Edl´en’s formulae” Metrologia, 35, pp. 133–139.
[10] Steele, A.G., Wood, B.,M., Douglas, R.,J., 2001, “Exclusive statistics: simple treatment of the unaviodable correlations from key comparison reference values,” Metrologia, 38, 483–488.
[11] Titov, I., Malinovsky, A., Balaidi, H., Franca, R.S., Massone, C.A., 1998, “Sub-nanometer Precision in Gauge Block Measurements,” CPEM 98, July 1998, Washington DC, USA.
[12] Comit´e Consultatif pour la D´efinition du M`etre, 3rd Session, 1962, pp. 18-19 and Proc`es-Verbaux CIPM, 52nd Session, 1963, pp. 26-27.
[13] Decker, J.E. and Pekelsky, J.R., 1997, “Uncertainty Evaluation for the Measurement of Gauge Blocks by Optical Interferometry,” Metrologia, 34, 479–493. [NRC Document No. 41374]
19
[14] Cox, M.G., 2002, “The evaluation of key comparison data,” Metrologia, 39, 589–595. [15] Beissner, K., 2002, “On a measure of consistency in comparison measurements,” Metrologia, 39, 59–63. [16] Beissner, K., 2003, “On a measure of consistency in comparison measurements: II Using effective degrees
of freedom,” Metrologia, 40, 31–35. [17] Kacker, R., Datla, R., Parr, A., 2002, “Combined result and associated uncertainty from interlaboratory
evaluations based on the ISO Guide,” Metrologia, 39, 279–293. [18] Bevington, P.,R., Data Reduction and Error Analysis for the Physical Sciences, (McGraw-Hill, New
York 1969) p. 84. [19] Birge, R.,T., 1932, “The Calculation of Errors by the Method of Least Squares,” Phys. Rev., 40, 207–227. [20] Taylor, B.,N., Parker, W.,H., Langenberg, D.,N., 1969, “Determination of e/h, QED, and the Funda-
mental Constants,” Rev. Mod. Phys., 41, 375–496. [21] Thalmann, R., 2002, “CCL key comparison: calibration of gauge blocks by interferometry,” Metrologia,
39, 165–177. [22] Guide to the Expression of Uncertainty in Measurement, 2nd ed., Geneva, International Organization for
Standardization, 1995.
20
A Evaluation of the Comparison Reference Value
The Key Comparison Reference Value (KCRV) xref and tables of equivalence are evaluated following some recommendations found in the metrology literature [14, 15, 16, 17]. The following discussion first considers the most reasonable mean value to apply for the SIM.4.2 Stage One KCRV, followed by tables of equivalence and bilateral equivalence.
A.1 Evaluations and Calculations
The inverse-variance weighted mean is evaluated by the following equation:
y=
n i=1
xi
u−2(xi)
n i=1
u−2
(xi)
(3)
where xi are the measurement results and u(xi) the standard uncertainties submitted by the comparison participants, and n represents the number of participants contributing to the evaluation of the mean. The standard uncertainty u(y) associated with y is
1
u(y) =
n i=1
u−2(xi)
.
(4)
The uncertainty in the arithmetic mean of equation (2) can be expressed as
1 u(x) = n
n
u2(xi).
(5)
i=1
Expression of equivalence di typically takes the form of the difference between the participant’s measured
value xi and xref .
di = xi − xref
(6)
The selection of the KCRV from amongst the inverse-variance weighted mean of equation (3), the simple arithmetic mean of equation (2), or the median usually depends on the overall consistency of the data sets. Taking an analogy to curve fitting, one has more confidence in a fit if the data does not contain outliers, or data that somehow creates a dominating influence. For the purpose of providing scientific evidence to support the CIPM-MRA, the inverse-variance weighted mean has the advantage that both participant measurement values and their evaluations of standard uncertainties are probed in the tests for consistency.
Statistical consistency of a comparison can be checked by evaluating the observed chi-squared value [14, 18]
χo2bs
=
n i=1
(xi − y)2 u2(xi)
.
(7)
The consistency check fails if
Pr{χ2(ν) > χ2obs} < 0.05.
(8)
For the seven participants in this part of the comparison, the degrees of freedom ν = 7 − 1 = 6. Values of
the calculated probabilities are listed in the Tables below. Similarly, if the variance weighted mean is a good representation of the data, then the value of the reduced chi-squared χ2ν = χo2bs/ν should be approximately
21
unity χ2ν = 1. With regard to the relative significance of these tests, it is important to consider that chi-squared tests are valid only if all participant distributions are Gaussian with mean value equal to the participant’s
stated value xi, and standard deviation equal to u(xi).
Another metric for evaluation of statistical consistency is the Birge Ratio [17, 19, 20], defined as
RB =
1 n−1
n
wi (xi − y)2
(9)
i=1
where the weights wi = 1/u2(xi) for i = 1, . . . , n are evaluated from the self-declared standard uncertainties. Consistency means that the results xi and the standard uncertainties u(xi) fit the Birge Ratio model, which in turn means that values of RB that are close to 1 or less suggest that the results of the comparison are consistent. Values of RB that are much greater than 1 suggest that results xi are inconsistent. Since the Birge Ratio calculation includes u(xi) as known parameters representing standard deviations of lab results xi, the Birge Ratio test requires that each of the uncertainties be reliable. When this assumption is not well justified, the conclusion of the Birge ratio test should not be taken too seriously. This warning, and the one
stated above for chi-squared, are particularly relevant considering the very low degrees of freedom stated for
INMETRO1 for short gauge block nominal lengths.
Another simple method to probe the consistency of a data set is to confirm that
|di| < k95 u(xi)
(10)
for all participants [15]. Values of di listed in the Tables apply the simple arithmetic mean as the reference value. In this comparison, the stated degrees of freedom from each participant do not rigorously allow for a coverage factor of k = 2 at a level of confidence of about 95 % for all laboratories. Therefore in the Tables below, individual k95 values are evaluated from the Student’s t-distribution taking into consideration the participant’s submitted νi.
The standard uncertainty u(di) in the stated equivalence di in the case where xref is evaluated by the arithmetic mean equation (2), and u(xref ) by equation (5) is expressed by [15] (see also [21]):
u(di) =
u2(xi)
+
u2(xref )
−
2 n
u2(xi).
(11)
This expression is used because it considers the correlation of each lab with the mean value. Even though it can be shown through exclusive statistics that the amount of correlation between participant labs and the simple arithmetic mean is small (see data plots in Figures 1 and 2 and discussion below) the more general approach is taken here.
The normalized deviation and its consistency limit are then calculated following
Ei
=
k
di u(di)
and
|Ei| < 1,
(12)
where k = 2 represents a confidence level of approximately 95 % that the measured value is within ±U of the true value (for a normal distribution). A more thorough approach [16] evaluates
E95,i
=
di k95(di) u(di)
and
|E95,i| < 1
(13)
where the self-declared degrees of freedom are used with the Welch-Satterthwaite equation to determine effective degrees of freedom for xref . Coverage factors k95(di) are evaluated from νeff (di) and the Student’s t-distribution (see [16, 22] for detail). Both versions of normalized deviation are tabulated with the other indicators of statistical consistency mentioned above.
22
100 mm Tungsten Carbide
NRC
di k95u(xi) −13 48
CENAM
18
71
INMETRO2 17
51
CEM
−18 26
INTI
−12 32
NIST
−1
35
INMETRO1 9
9
|E95,i| = di/k95(di)u(di) 0.30 0.29 0.37 0.67 0.39 0.03 0.53
Median
Simple arithmetic mean:
standard uncertainty
Variance weighted mean:
standard uncertainty
Variance weighted mean sans INMETRO1:
standard uncertainty
Observed chi-squared
Observed chi-squared sans INMETRO1
Degrees of freedom:
Pr{χ2(ν) > χ2obs} Pr{χ2(ν) > χ2obs} sans INMETRO1
Reduced chi-squared
sans INMETRO1
Birge Ratio
sans INMETRO1
|Ei| = di/2u(di) 0.30 0.29 0.36 0.66 0.38 0.03 0.52
−46.0 nm −44.9 nm
8.2 nm −39.5 nm
3.6 nm −53.3 nm
7.7 nm 6.4 2.3 6 0.38 0.89 1.07 0.38 1.03 0.62
23
50 mm Tungsten Carbide
NRC
di k95u(xi)
5
35
CENAM
17
43
INMETRO2 6
36
CEM
−13 20
INTI
−6
24
NIST
−2
26
INMETRO1 −10
9
|E95,i| = di/k95(di)u(di) 0.17 0.46 0.20 0.63 0.24 0.06 0.81
Median
Simple arithmetic mean:
standard uncertainty
Variance weighted mean:
standard uncertainty
Variance weighted mean sans INMETRO1:
standard uncertainty
Observed chi-squared
Observed chi-squared sans INMETRO1
Degrees of freedom:
Pr{χ2(ν) > χ2obs} Pr{χ2(ν) > χo2bs} sans INMETRO1
Reduced chi-squared
sans INMETRO1
Birge Ratio
sans INMETRO1
|Ei| = di/2u(di) 0.18 0.46 0.20 0.62 0.24 0.06 0.80
−32.0 nm −30.4 nm
5.6 nm −38.8 nm
2.4 nm −33.5 nm
5.7 nm 3.4 2.4 6 0.75 0.88 0.57 0.40 0.76 0.63
24
20 mm Tungsten Carbide
NRC
di k95u(xi)
1
31
CENAM
12
30
INMETRO2 −8
32
CEM
8
18
INTI
−11 22
NIST
9
21
INMETRO1 −14
9
|E95,i| = di/k95(di)u(di) 0.05 0.46 0.27 0.48 0.52 0.48 1.43
Median
Simple arithmetic mean:
standard uncertainty
Variance weighted mean:
standard uncertainty
Variance weighted mean sans INMETRO1:
standard uncertainty
Observed chi-squared
Observed chi-squared sans INMETRO1
Degrees of freedom:
Pr{χ2(ν) > χ2obs} Pr{χ2(ν) > χo2bs} sans INMETRO1
Reduced chi-squared
sans INMETRO1
Birge Ratio
sans INMETRO1
|Ei| = di/2u(di) 0.06 0.45 0.28 0.48 0.51 0.47 1.42
10.0 nm 8.6 nm 4.5 nm −2.6 nm 1.8 nm 11.5 nm 4.8 nm
13.1 3.1 6 0.04 0.79 2.18 0.52 1.48 0.72
25
8 mm Tungsten Carbide
NRC
di k95u(xi)
8
31
CENAM
17
28
INMETRO2 −3 30
CEM
2
16
INTI
−9 22
NIST
−6 19
INMETRO1 −8
7
|E95,i| = di/k95(di)u(di) 0.28 0.67 0.12 0.11 0.46 0.35 0.85
Median
Simple arithmetic mean:
standard uncertainty
Variance weighted mean:
standard uncertainty
Variance weighted mean sans INMETRO1:
standard uncertainty
Observed chi-squared
Observed chi-squared sans INMETRO1
Degrees of freedom:
Pr{χ2(ν) > χo2bs} Pr{χ2(ν) > χ2obs} sans INMETRO1
Reduced chi-squared
sans INMETRO1
Birge Ratio
sans INMETRO1
|Ei| = di/2u(di) 0.31 0.67 0.13 0.11 0.45 0.34 0.84
34.0 nm 37.2 nm 4.2 nm 30.9 nm 1.9 nm 37.1 nm 4.5 nm
5.3 3.0 6 0.50 0.81 0.89 0.49 0.94 0.70
26
5 mm Tungsten Carbide
NRC
di k95u(xi)
2
31
CENAM
18
28
INMETRO2 −1 30
CEM
−4 16
INTI
−7 22
NIST
2
18
INMETRO1 −8
1
|E95,i| = di/k95(di)u(di) 0.06 0.70 0.05 0.28 0.37 0.09 0.90
Median
Simple arithmetic mean:
standard uncertainty
Variance weighted mean:
standard uncertainty
Variance weighted mean sans INMETRO1:
standard uncertainty
Observed chi-squared
Observed chi-squared sans INMETRO1
Degrees of freedom:
Pr{χ2(ν) > χo2bs} Pr{χ2(ν) > χ2obs} sans INMETRO1
Reduced chi-squared
sans INMETRO1
Birge Ratio
sans INMETRO1
|Ei| = di/2u(di) 0.06 0.70 0.06 0.28 0.36 0.09 0.89
16.0 nm 17.4 nm 4.2 nm 10.0 nm 0.6 nm 17.0 nm 4.4 nm
4.9 2.4 6 0.56 0.88 0.81 0.40 0.90 0.63
27
2 mm Tungsten Carbide
NRC
di k95u(xi)
1
31
CENAM
11
27
INMETRO2 3
30
CEM
11
16
INTI
−9
22
NIST
−12 18
INMETRO1 −7
5
|E95,i| = di/k95(di)u(di) 0.05 0.46 0.12 0.72 0.43 0.68 0.75
Median
Simple arithmetic mean:
standard uncertainty
Variance weighted mean:
standard uncertainty
Variance weighted mean sans INMETRO1:
standard uncertainty
Observed chi-squared
Observed chi-squared sans INMETRO1
Degrees of freedom:
Pr{χ2(ν) > χ2obs} Pr{χ2(ν) > χ2obs} sans INMETRO1
Reduced chi-squared
sans INMETRO1
Birge Ratio
sans INMETRO1
|Ei| = di/2u(di) 0.05 0.46 0.13 0.71 0.43 0.67 0.74
−10.0 nm −11.3 nm
4.2 nm −16.6 nm
1.9 nm −10.4 nm
4.4 nm 7.4 5.0 6 0.28 0.55 1.23 0.83 1.11 0.91
28
100 mm Steel
di
NRC
−19
CENAM
12
INMETRO2 37
CEM
−43
INTI
1
NIST
5
INMETRO1 7
k95u(xi) 52 45 57 33 41 35 9
|E95,i| = di/k95(di)u(di) 0.41 0.29 0.73 1.33 0.02 0.15 0.43
Median
Simple arithmetic mean:
standard uncertainty
Variance weighted mean:
standard uncertainty
Variance weighted mean sans INMETRO1:
standard uncertainty
Observed chi-squared
Observed chi-squared sans INMETRO1
Degrees of freedom:
Pr{χ2(ν) > χo2bs} Pr{χ2(ν) > χ2obs} sans INMETRO1
Reduced chi-squared
sans INMETRO1
Birge Ratio
sans INMETRO1
|Ei| = di/2u(di) 0.41 0.28 0.72 1.31 0.02 0.14 0.42
−100.0 nm −104.9 nm
8.0 nm −100.4 nm
3.8 nm −111.9 nm
8.6 nm 10.5 8.3
6 0.11 0.22 1.75 1.38 1.32 1.17
29
50 mm Steel
di k95u(xi) |E95,i| = di/k95(di)u(di)
NRC
0
37
0.01
CENAM
5
25
0.20
INMETRO2 27
38
0.79
CEM
−22 22
1.06
INTI
−12 28
0.48
NIST
5
26
0.19
INMETRO1 −1
6
0.11
Median
Simple arithmetic mean:
standard uncertainty
Variance weighted mean:
standard uncertainty
Variance weighted mean sans INMETRO1:
standard uncertainty
Observed chi-squared
Observed chi-squared sans INMETRO1
Degrees of freedom:
Pr{χ2(ν) > χ2obs} Pr{χ2(ν) > χ2obs} sans INMETRO1
Reduced chi-squared
sans INMETRO1
Birge Ratio
sans INMETRO1
|Ei| = di/2u(di) 0.01 0.19 0.79 1.04 0.47 0.19 0.11
31.0 nm 31.3 nm 5.3 nm 29.7 nm 2.3 nm 27.4 nm 5.7 nm
6.8 6.6 6 0.34 0.35 1.14 1.11 1.07 1.05
30
10 mm Steel
di k95u(xi) |E95,i| = di/k95(di)u(di)
NRC
14
31
0.52
CENAM
1
15
0.09
INMETRO2 30
30
1.14
CEM
−25 18
1.46
INTI
−7
22
0.33
NIST
−13 19
0.71
INMETRO1 −2
7
0.23
Median
Simple arithmetic mean:
standard uncertainty
Variance weighted mean:
standard uncertainty
Variance weighted mean sans INMETRO1:
standard uncertainty
Observed chi-squared
Observed chi-squared sans INMETRO1
Degrees of freedom:
Pr{χ2(ν) > χo2bs} Pr{χ2(ν) > χ2obs} sans INMETRO1
Reduced chi-squared
sans INMETRO1
Birge Ratio
sans INMETRO1
|Ei| = di/2u(di) 0.57 0.09 1.22 1.44 0.33 0.69 0.23
18.7 nm 20.7 nm 3.9 nm 18.5 nm 1.5 nm 16.7 nm 4.1 nm
14.6 14.4
6 0.02 0.03 2.43 2.40 1.56 1.55
31
8 mm Steel
NRC
di k95u(xi) |E95,i| = di/k95(di)u(di)
−20 31
0.73
CENAM
−4
14
0.28
INMETRO2 32
30
1.21
CEM
−2
16
0.13
INTI
−9
22
0.45
NIST
10
19
0.57
INMETRO1 −7
4
0.85
Median
Simple arithmetic mean:
standard uncertainty
Variance weighted mean:
standard uncertainty
Variance weighted mean sans INMETRO1:
standard uncertainty
Observed chi-squared
Observed chi-squared sans INMETRO1
Degrees of freedom:
Pr{χ2(ν) > χo2bs} Pr{χ2(ν) > χ2obs} sans INMETRO1
Reduced chi-squared
sans INMETRO1
Birge Ratio
sans INMETRO1
|Ei| = di/2u(di) 0.80 0.28 1.29 0.13 0.45 0.56 0.85
45.0 nm 49.0 nm 3.8 nm 43.1 nm 1.5 nm 48.8 nm 3.9 nm
11.8 9.4 6 0.07 0.15 1.97 1.56 1.41 1.25
32
5 mm Steel
di k95u(xi) |E95,i| = di/k95(di)u(di)
NRC
1
31
0.05
CENAM
2
14
0.17
INMETRO2 22
30
0.84
CEM
−12 16
0.76
INTI
−12 22
0.58
NIST
1
18
0.08
INMETRO1 −4
7
0.51
Median
Simple arithmetic mean:
standard uncertainty
Variance weighted mean:
standard uncertainty
Variance weighted mean sans INMETRO1:
standard uncertainty
Observed chi-squared
Observed chi-squared sans INMETRO1
Degrees of freedom:
Pr{χ2(ν) > χ2obs} Pr{χ2(ν) > χ2obs} sans INMETRO1
Reduced chi-squared
sans INMETRO1
Birge Ratio
sans INMETRO1
|Ei| = di/2u(di) 0.05 0.17 0.90 0.75 0.58 0.08 0.51
−52.0 nm −53.4 nm
3.8 nm −57.1 nm
1.5 nm −54.8 nm
3.9 nm 6.2 5.8 6 0.40 0.45 1.04 0.97 1.02 0.98
33
2 mm Steel
NRC
di k95u(xi) |E95,i| = di/k95(di)u(di)
−18 31
0.66
CENAM
1
14
0.07
INMETRO2 24
30
0.90
CEM
−10 16
0.65
INTI
−3
22
0.15
NIST
6
18
0.36
INMETRO1 0
7
0.01
Median
Simple arithmetic mean:
standard uncertainty
Variance weighted mean:
standard uncertainty
Variance weighted mean sans INMETRO1:
standard uncertainty
Observed chi-squared
Observed chi-squared sans INMETRO1
Degrees of freedom:
Pr{χ2(ν) > χ2obs} Pr{χ2(ν) > χ2obs} sans INMETRO1
Reduced chi-squared
sans INMETRO1
Birge Ratio
sans INMETRO1
|Ei| = di/2u(di) 0.72 0.07 0.97 0.64 0.15 0.35 0.01
35.9 nm 36.0 nm 3.8 nm 35.8 nm 1.5 nm 35.1 nm 3.9 nm
6.7 6.6 6 0.35 0.36 1.11 1.11 1.05 1.05
Table 12: List of median, simple arithmetic mean, variance weighted mean and statistical consistency parameters chi-squared, Birge Ratio and normalized deviations |Ei| and |E95,i|.
34
A.2 Discussion
Consistency tests of chi-squared, Birge ratio and Pr{χ2(ν) > χo2bs} probablility are evaluated based on the inverse variance weighted mean, keeping in mind that these tests are designed to test statistical consistency only in the case of the variance weighted mean. Values for these parameters are listed in Tables below and plotted in Figure 7. For the case of mean value evaluated with equal weights (arithmetic mean), sophisticated consistency indicators must be evaluated by other methods. This Appendix reports on the simple comparison of di with the participant’s claimed U95,i.
The 10 mm steel gauge block data set could be considered discrepant. At the time of writing this report, results from pilot measurements over the time duration of Stage Two of the comparison show that this gauge block was shrinking during the time of this comparison. However at the time when Stage One was just completed, it was not obvious from the pilot measurements taken during the slice of time duration for Stage One of the comparison that the gauge block was indeed shrinking. For this reason, results from the 10 mm steel gauge block are left out of the discussions of statistical consistency.
Now the technique of exclusive statistics [10] provides a simple and statistically rigorous procedure for demonstrating the amount of correlation of each lab with the evaluated mean. The ‘exclusive mean’ is the mean value (arithmetic or otherwise) evaluated for each participant in turn, omitting the participants own value from the calculation. The exclusive mean value includes the values submitted by each of the other participants, but excludes the value of the ‘exclusive mean participant’. The exclusive mean expresses the results of a comparison from the point of view of how each laboratory performs with respect to the rest of the world. The plots of the data in the body of the report (Figures 1 and 2) show the simple arithmetic mean evaluated from all participants as a solid line. The exclusive simple arithmetic mean is plotted as the short thick lines for each participant. The difference between the inclusive and exclusive means allows us to graphically observe correlations between individual labs and the mean value. For the simple arithmetic mean case, it is clear that none of the individual labs have a dominating influence on the mean value. However, one can observe strong correlation between INMETRO1 and the inverse variance weighted mean because of the relatively small measurement uncertainties claimed by INMETRO1. The 5 mm tungsten carbide and 20 mm tungsten carbide gauges are the worst-case examples; they are shown in Figure 6. In general, INMETRO1 results dominate the inverse variance weighted mean by a relative amount ranging from 78 % to 98 %.
Reduced chi-squared, Birge Ratio and Pr{χ2(ν) > χo2bs} are evaluated both including all participants and a second time excluding the results of INMETRO1 for the reason that INMETRO1 submitted very optimistic uncertainty claims relative to conventional capabilities3. For most gauge blocks, these consistency tests are either improved or remain the same when INMETRO1 results are included (see below). The measurement values themselves cannot be considered to be outliers since inclusion or exclusion of xINMETRO1 for any of the gauge blocks of the comparison does not change the arithmetic mean or median values significantly. This comparison analysis at once probes the impact of the small standard uncertainties and low degrees of freedom submitted by INMETRO1 on the KCRV, and attempts to settle on a KCRV that represents all participants fairly.
The Tables list the simple consistency indicator of comparing di evaluated with the simple arithmetic mean value with each participants submitted U95. Examination of di < k95u(xi) largely passes for all participants for all gauge blocks with few discrepancies considering the expected 5 % based on statistics. However, di < k95u(xi) fails for 6 out of the 11 gauge blocks for INMETRO1, yet in all except one case INMETRO1 passes the traditional normalized deviation |Ei| and |E95,i|. Interestingly, the extra precision offered by the evaluation of |E95,i| does not result in appreciable differences from the result of the approximate |Ei|.
3The technical debate as to the validity of INMETRO1 technique and uncertainty analysis is deferred to discussion in the literature.
35
Figure 6: Plots of central length expressed as deviation from nominal length reported by each participant for 20 mm and 5 mm tungsten carbide gauge blocks. Thick error bars represent the standard uncertainty, while longer thin error bars represent k95u(xi) where k95 = tp(νi) from the Student’s t-distribution for standard uncertainties u(xi) and degrees of freedom νi submitted by the participants. The solid line represents the inverse variance weighted mean evaluated taking results of all participants. The dash for each participant represents the exclusive inverse variance weighted mean (see text).
36
For most steel gauge blocks, the chi-squared and Birge Ratio values remain much the same whether or not INMETRO1 results are included. Although the 100 mm steel and the 8 mm steel gauge blocks are on the verge of discrepant, they are somewhat improved with the exclusion of INMETRO1 results. Values of Pr{χ2(ν) > χo2bs} are generally improved with INMETRO1 excluded. The results for tungsten carbide gauge blocks appear to be more sensitive to the weighting influence of INMETRO1. In general, larger differences between weighted mean and simple arithmetic mean and the consistency parameters is observed. For the two gauge blocks in Figure 6, INMETRO1 results could be considered as outliers relative to the variance weighted mean evaluated for the sub-set of the participants excluding INMETRO1 (the exclusive variance weighted mean). The 20 mm tungsten carbide results could be considered discrepant; and this gauge block was not changing length during the comparison. The INMETRO1 result for this gauge is 4 standard deviations away from the exclusive variance weighted mean value, and the chi-squared, Birge ratio and probablility results reflect that the data sub-set would be more consistent than the full data set containing all participants. For the 5 mm gauge block the INMETRO1 result is about 11 standard deviations away from the exclusive variance weighted mean, and the reduced chi-squared and Birge ratio parameters indicate that the full data set including all participants is more consistent. This is because the INMETRO1 result with the very small measurement uncertainty ‘owns’ the variance weighted mean, and is therefore very consistent with it. The weighted mean is not recommended as the KCRV when not all self-declared uncertainties are considered reliable [17]. The very low degrees of freedom reported by INMETRO1, and the dominating weight of their results to the variance weighted mean as demonstrated by exclusive statistics and the behaviour of the consistency parameters, provide reasonable evidence to select the simple arithmetic mean for the SIM.4.2 Stage One KCRV.
37
Figure 7: Graphical representation of mean values and consistency parameters evaluated for the comparison data.
38
Participant
NRC CENAM INMETRO2 INMETRO1
INTI NIST CEM
2
−18 ± 25 1 ± 14 24 ± 25 0±8 −3 ± 20 6 ± 17
−10 ± 16
Nominal Length of Steel Gauge Blocks /mm
5
8
10
50
1 ± 25 −20 ± 25 14 ± 25 0 ± 32
2 ± 14 −4 ± 14 1 ± 15 5 ± 24
22 ± 25 −4 ± 8
32 ± 25 −7 ± 8
30 ± 25 −2 ± 8
27 ± 34 −1 ± 11
−12 ± 20 −9 ± 20 −7 ± 20 −12 ± 26
1 ± 18 10 ± 18 −13 ± 18 5 ± 25
−12 ± 16 −2 ± 16 −25 ± 17 −22 ± 21
100
−19 ± 47 12 ± 42 37 ± 52 7 ± 18 1 ± 39 5 ± 34 −43 ± 33
Table 13: Difference between participant value and the KCRV (simple arithmetic mean) listed with the k = 2 expanded uncertainty U (di) for steel gauge blocks.
Participant
NRC CENAM INMETRO2 INMETRO1
INTI NIST CEM
Nominal Length of Tungsten Carbide Gauge Blocks /mm
2
5
8
20
50
100
1 ± 25 11 ± 25 3 ± 25 −7 ± 9 −9 ± 20 −12 ± 18 11 ± 16
2 ± 25 18 ± 25 −1 ± 25 −8 ± 8 −7 ± 20 2 ± 18 −4 ± 16
8 ± 25 17 ± 25 −3 ± 25 −8 ± 9 −9 ± 20 −6 ± 18 2 ± 16
1 ± 25 12 ± 28 −8 ± 27 −14 ± 10 −11 ± 21 9 ± 20 8 ± 18
5 ± 31 17 ± 38 6 ± 32 −10 ± 12 −6 ± 23 −2 ± 25 −13 ± 20
−13 ± 44 18 ± 63 17 ± 47 9 ± 18 −12 ± 32 −1 ± 34 −18 ± 27
Table 14: Difference between the participant value and the KCRV (simple arithmetic mean) listed with the k = 2 expanded uncertainty U (di) for tungsten carbide gauge blocks.
B Degrees of Equivalence
Tables of degrees of equivalence list the difference between the measurement value submitted by each participant and the KCRV as described by equation (6) with xref ≡ x. The expanded uncertainty U (di) = 2u(di) where u(di) is calculated from equation (11) for the n = 7 participants of the comparison. On the basis of statistical variability alone, 5 % of measurements would be expected to be classified as discrepant. Values in Tables 13 and 14, and the above discussion allude to the suggestion that most participant labs could have been conservative in their estimate of uncertainties.
39
C Linking of SIM.4.2 Gauge Block Comparison to the CCL-K1 Gauge Block Comparison
At its 11th meeting in 2003, the Consultative Committee for Length (CCL) decided that “artefact-based key comparisons in dimensional metrology will not use a numerical link between a CCL key comparison and any corresponding RMO comparison. Instead, the link will be based on competencies demonstrated by the participant laboratory which took part as linking NMIs in the CCL and RMO key comparisons. The CCL and RMO key comparisons will be deemed as being equivalent.” If the linking NMIs were judged to have performed competently in both comparisons (CCL, RMO), then the comparisons were to be regarded as equivalent. The judgment of the competence is the responsibility of the WGDM upon consideration of the Draft B report. The Sistema Interamericano de Metrolog´ia (SIM) Regional Comparison of gauge block calibration by interferometry SIM.4.2 links to the CCL Key Comparison CCL-K1 [21] through the participation of the following national metrology institutes: National Research Council Canada Institute for National Measurement Standards (NRC-INMS), the United States National Institute of Standards and Technology (NIST) and the Centro Nacional de Metrolog´ıa (CENAM) of Mexico.
D Tables of Bilateral Equivalence
The degree of equivalence between institute i and institute j is listed as a pair of values where
di,j = xi − xj
(14)
and
U95(di,j ) = k95 u(di,j )
(15)
where u2(di,j) = u2(xi) + u2(xj), and k95 is evaluated from the Student’s t-distribution and the effective degrees of freedom νeff determined by the Welch-Satterthwaite approximation from νi and νj submitted by the participants according to Section G of the GUM [22].
40
NRC CENAM INMETRO2 INMETRO1
INTI NIST CEM
NRC
−19 ± 33 −42 ± 41 −18 ± 31 −15 ± 37 −24 ± 35 −8 ± 34
CENAM
−23 ± 33 1 ± 14 4 ± 26 −5 ± 23 11 ± 21
INMETRO2
24 ± 30 27 ± 36 18 ± 34 34 ± 33
INMETRO1
3 ± 22 −6 ± 18 10 ± 16
INTI
−9 ± 28 7 ± 27
NIST 16 ± 24
CEM
Table 15: Bilateral equivalence di,j ± k95u(di,j) for 2 mm steel gauge block.
NRC CENAM INMETRO2 INMETRO1
INTI NIST CEM
NRC
−1 ± 33 −21 ± 41
6 ± 31 13 ± 37 0 ± 35 13 ± 34
CENAM
−20 ± 33 7 ± 14 14 ± 26 1 ± 23 14 ± 21
INMETRO2
27 ± 30 34 ± 36 21 ± 35 34 ± 33
INMETRO1
8 ± 22 −6 ± 19 8 ± 16
INTI
−13 ± 29 0 ± 27
NIST 13 ± 24
CEM
Table 16: Bilateral equivalence di,j ± k95u(di,j) for 5 mm steel gauge block.
NRC CENAM INMETRO2 INMETRO1
INTI NIST CEM
NRC
−16 ± 33 −52 ± 41 −13 ± 31 −11 ± 37 −30 ± 35 −18 ± 34
CENAM
−36 ± 33 3 ± 15 5 ± 26
−14 ± 24 −2 ± 21
INMETRO2
39 ± 30 41 ± 36 22 ± 35 34 ± 33
INMETRO1
2 ± 22 −17 ± 19 −5 ± 16
INTI
−19 ± 29 −7 ± 27
NIST 12 ± 25
CEM
Table 17: Bilateral equivalence di,j ± k95u(di,j) for 8 mm steel gauge block.
NRC CENAM INMETRO2 INMETRO1
INTI NIST CEM
NRC
13 ± 33 −16 ± 41 16 ± 31 21 ± 37 27 ± 35 39 ± 35
CENAM
−29 ± 33 3 ± 15 8 ± 26 14 ± 24 26 ± 23
INMETRO2
32 ± 30 37 ± 36 43 ± 35 55 ± 34
INMETRO1
5 ± 22 11 ± 19 23 ± 18
INTI
6 ± 29 18 ± 28
NIST 12 ± 26
CEM
Table 18: Bilateral equivalence di,j ± k95u(di,j) for 10 mm steel gauge block.
41
NRC CENAM INMETRO2 INMETRO1
INTI NIST CEM
NRC
−5 ± 47 −27 ± 52
1 ± 37 12 ± 45 −5 ± 45 22 ± 42
CENAM
−22 ± 46 6 ± 26 17 ± 37 0 ± 36 27 ± 33
INMETRO2
28 ± 39 39 ± 47 22 ± 46 49 ± 44
INMETRO1
11 ± 28 −6 ± 27 21 ± 22
INTI
−17 ± 38 10 ± 35
NIST 27 ± 34
CEM
Table 19: Bilateral equivalence di,j ± k95u(di,j) for 50 mm steel gauge block.
NRC CENAM INMETRO2 INMETRO1
INTI NIST CEM
NRC
−31 ± 73 −56 ± 77 −27 ± 52 −20 ± 66 −24 ± 62 24 ± 61
CENAM
−25 ± 72 5 ± 46 11 ± 61 7 ± 57 55 ± 56
INMETRO2
30 ± 58 36 ± 70 32 ± 67 80 ± 66
INMETRO1
7 ± 42 3 ± 36 51 ± 34
INTI
−4 ± 54 44 ± 53
NIST 48 ± 48
CEM
Table 20: Bilateral equivalence di,j ± k95u(di,j) for 100 mm steel gauge block.
NRC CENAM INMETRO2 INMETRO1
INTI NIST CEM
NRC
−10 ± 42 −2 ± 41 8 ± 31 10 ± 37 13 ± 35 −10 ± 34
CENAM
8 ± 40 18 ± 28 20 ± 35 23 ± 33 0 ± 32
INMETRO2
10 ± 31 12 ± 36 15 ± 34 −8 ± 33
INMETRO1
2 ± 22 5 ± 18 −18 ± 16
INTI
3 ± 28 −20 ± 27
NIST −23 ± 24
CEM
Table 21: Bilateral equivalence di,j ± k95u(di,j) for 2 mm tungsten carbide gauge block.
NRC CENAM INMETRO2 INMETRO1
INTI NIST CEM
NRC
−16 ± 42 3 ± 41 9 ± 31 9 ± 37 0 ± 35 6 ± 34
CENAM
19 ± 40 25 ± 28 25 ± 35 16 ± 33 22 ± 32
INMETRO2
6 ± 30 6 ± 36 −3 ± 35 3 ± 33
INMETRO1
0 ± 22 −9 ± 18 −3 ± 16
INTI
−9 ± 29 −3 ± 27
NIST 6 ± 24
CEM
Table 22: Bilateral equivalence di,j ± k95u(di,j) for 5 mm tungsten carbide gauge block.
42
NRC CENAM INMETRO2 INMETRO1
INTI NIST CEM
NRC
−9 ± 42 11 ± 41 16 ± 31 17 ± 37 14 ± 35 6 ± 34
CENAM
20 ± 40 25 ± 28 26 ± 35 23 ± 33 15 ± 32
INMETRO2
5 ± 31 6 ± 36 3 ± 35 −5 ± 33
INMETRO1
2 ± 22 −2 ± 19 −10 ± 16
INTI
−3 ± 29 −11 ± 27
NIST −8 ± 25
CEM
Table 23: Bilateral equivalence di,j ± k95u(di,j) for 8 mm tungsten carbide gauge block.
NRC CENAM INMETRO2 INMETRO1
INTI NIST CEM
NRC
−11 ± 44 9 ± 42 15 ± 31 12 ± 36 −8 ± 36 −7 ± 34
CENAM
20 ± 43 26 ± 31 23 ± 37 3 ± 37 4 ± 35
INMETRO2
6 ± 32 3 ± 38 −17 ± 37 −16 ± 36
INMETRO1
−3 ± 22 −23 ± 21 −22 ± 18
INTI
−20 ± 30 −19 ± 28
NIST 1 ± 27
CEM
Table 24: Bilateral equivalence di,j ± k95u(di,j) for 20 mm tungsten carbide gauge block.
NRC CENAM INMETRO2 INMETRO1
INTI NIST CEM
NRC
−12 ± 58 −1 ± 50 15 ± 35 11 ± 42 7 ± 43 18 ± 40
CENAM
11 ± 55 27 ± 43 23 ± 49 19 ± 50 30 ± 47
INMETRO2
16 ± 37 12 ± 43 8 ± 45 19 ± 41
INMETRO1
−4 ± 24 −8 ± 27 3 ± 21
INTI
−4 ± 35 7 ± 31
NIST 11 ± 33
CEM
Table 25: Bilateral equivalence di,j ± k95u(di,j) for 50 mm tungsten carbide gauge block.
NRC CENAM INMETRO2 INMETRO1
INTI NIST CEM
NRC
−31 ± 91 −30 ± 70 −22 ± 49 −1 ± 57 −12 ± 59
5 ± 54
CENAM
1 ± 87 9 ± 71 30 ± 77 19 ± 79 36 ± 75
INMETRO2
8 ± 52 29 ± 60 18 ± 62 35 ± 57
INMETRO1
21 ± 32 10 ± 36 27 ± 27
INTI
−11 ± 47 6 ± 40
NIST 17 ± 43
CEM
Table 26: Bilateral equivalence di,j ± k95u(di,j) for 100 mm tungsten carbide gauge block.
43
Ver+/-
