COMBINING INDEPENDENT P-VALUES INDEPENDENT P-VALUES BY F. W ... nonparametric setting and was...
-
Upload
hoangxuyen -
Category
Documents
-
view
220 -
download
1
Transcript of COMBINING INDEPENDENT P-VALUES INDEPENDENT P-VALUES BY F. W ... nonparametric setting and was...
COMBINING INDEPENDENT P-VALUES
BYt
F. W. SCHOLZ
TECHNICAL REPORT NO. 11
SEPTEMBER 1981
DEPARTMENT OF STATISTICSUNIVERSITY OF WASHINGTONSEATTLE) WASHINGTON 98195
i or Sta s cal Anal t wian A sociate ~rnTOC
Statistics at iversi
<: ect ca 6-.J
6 15 6
ABSTRACT
In combining P-values P1, •.. , Pk of k independent tests of hypothesis we findthat the combining statistics of several proposals are of the form T(h =
2:h(Pi)' where h is some nonincreasing function on (0,1), the joint hypothesisbeing rejected when T(h) is too large. We propose to let the P-values themselveschoose the "proper" function h by taking that h (suitably standar-d i zed) which
yields the largest value of T(h) which we denote by Sk' It is shown that Sk isquite easily computed. Its null distribution is derived for k = 2, 3 and
simulated for k =4, ... , 10. A limited power comparison between S2 and Fisher'stest is presented.
P-values, rical dist bu on on,acen i ator al
O. Introduction
Although Fisher's method is certainly the most popular for combining P-values of
independent statistical tests, other proposals, some recent and others not so
recent, have been put forward. For reviews and references on this subject see
Birnbaum (1954), Oosterhoff (1969), Koziol and Perlman (1978), Mudholkar and
George (1979).
It appears that many of these proposals combine the P-values P1, ... , Pk through
a statistic of the following general form:
where h is some decreasing function defined on the interval (0,1). The joint
hypothesis is then rejected if T(h) is too large. As illustration we mention a
few forms for h:
i) h(x) = -2 log x (Fisher, 1932)
ii) h(x) = _<:>-1(x) (Liptak, 1958), here <:>-1 is the inverse of the standard
normal distribution function.
iii) h(x) = -log(x/(l-x)) Mudholkar and Georqe (1979)
Even Tippet s 1931 combining statistics min(P1' •.. , Pk) may in a limiti
sense be subsumed in this wide class of test statistics T(h) as was pointed
to us by M. Perlman; namely consider hr(x) = x-r r > 0 and instead of T(hr).consider T(hr)l/r with r-oo to obtain the reciprocal of Tippet's statistic.
Thus we have a fairly wide class {T(h): h +} of combining test statistics and
each member of this class will yield a most powerful test against some particular
alternative, cf Birnbaum (1954). These alternatives may be quite obscure for
each individual T(h) and certainly are not the main motivation for choosing one
form of T(h) over another. If we did know the alternatives quite specifically we
could deal with the problem via the Neyman Pearson theory. Often, however, the
alternative is specified only vaguely, i.e. one or more of the P-values will te~d
to be stochastically small. We propose to let the P-values "choose the proper"
score function h, namely by selecting that h for which T(h) yields the most
extreme value. For this to be meaningful requires that the functions h be
standardized appropriately, aside from being nonincreasing. This proposal is
nothing but an appl ication of Roy's (1953) union intersection principle to a
nonparametric setting and was inspired by a similar proposal of Behnen (1975) in
the case of two sample linear rank statistics.
Section 1 defines the new combining statistic in detail and presents alternate
f'orms which are amenable to computation. Section 2 examines the null df st r i-
bution of the test statistic, and Section 3 gives a limited examination of its
power behavior in comparison to Fisher's test.
1. Derivation the Test Statistic and its Computation
let H be the class of all real-va ued functions h defined on the interva
es:
i) h is nonincreasing on (Ot1)
.. \11}
1
Jh(x)dx = 0 t and iii)o
Given the observed P-values PIt .•• t Pk from k independent statistical tests,
define the following combination statistic:
(1.1 )
The statistic Sk given in the form (1.1) is not amenable to computation.
Fortunately it turns out that Sk may be computed quite easily once we give its
alternative equivalent representation. To do so we first introduce some nota-
tion.
Let Fk(·) be the empirical cumulative distribution function of PI' ..• , Pk and1\
let Fk(·) be the smallest concave majorant to Fk(·) on the interval (0,1).1\ 1\
Fur-ther, let f k(· r be the left continuous density of Fk(.). The following
proposition represents Sk in a form which may easily be computed as we will
elaborate later.
Proposition 1: Sk' k (J (fk (Xl- l l2dX) Is • :k Ilfk-1112
o
1\Proof: Since Fk is stochastically sma ler than Fk, we have for nonincreasi
(cf. Lehmann 1959
k<
1\
is constant between verteces where Fk
For h E H we further have by the Cauchy Schwarz inequality:
fl.3
* 1\ 1\Note that h (x}: = (fk(x)-1)/llfk-1112 E H and that we have equality in 0.3
* *h(x) = h (x) almost surely. Since h
*touches Fk, one easily sees, taking into account the left continuity of h , that
(1. 4)
Thus (1.2) - (1.4) imply the desired result.
This simple proof of Proposition 1 is, in major parts, due to Piet ~roeneboom and
replaces our earlier, lengthier version which consisted of a modification of
Behnen's argument. Of course Behnen's proof may be shortened similarly. Having
obtained a much more manageable form for Sk' let us give another representation
which outlines a computational algorithm.
i = 1, ... , k be some weights.
the first k(p = 0) i = 1\ 0 ' ••• , k
First denote by c, : p('\-Pr' , \1,0 1; ,1-.;.)
spacings of the ordered Pi and let di,o = 1,
If the c;,o are monotonically increasing then
I
k 2k2 ff2 x dx = C i ,0 where ko = k.
0 = l
If the ci,o are not monotonically increasing, we smooth them out by the following
Pool-Adjacent-Violator algorithm (cf. Barlow et al. 1972): If after the r t h step
in the algorithm we still have
di r, <ci r,
set
for j =1, ••. , i-1
c i , r+1 = c," r + ci+1 r ' di r+1, , , = di r + di +1 r, ,
= k -1.r
for j = ; + 1, .", kr-1
We repeat this process until we finally have
for some s ~ o.
One easily s~es that di,s/{koci,s) represents the slope of the it h
linear segment1\
of Fk, Hence
where
ks 2Rk = ~
di,sG:l ci,s
1\An alternate computational expression for fk(x) may be found in Barlow et al.
(1972) p. 225.
The following are explicit computational formulae for R2 and R3:
=
where c1 = P(1) and c2 = P(2) - P(1) and P(1)< P(2) are the ordered P-values.
=
where Cl = PI)' c2 = P 2) - P 1 and c3 = P 3 - P 2 and PI) < P(2 < 0 3
the ordered P-values.
are
2. The Distribution of Sk Under the Hypothesis
Under the hypothesis the o-values PI' ... , Pk are distributed uniformly on the
interval (0,1). The first k spacings Ci = p(i) - P(i-l) (p(o)= 0) i = 1, •.. , k
are distributed uniformly over the simplex
kc. > 0 'ri., L c" < I}, , i=l
(cf. Pyke i965).
From this one may derive expressions for P(Rk $ r) by tedious but straightforward
integration. The resulting expressions for k = 2 and k =3 are given below, the
latter could not be reduced beyond its integral form. For k > 3 one encounters
higher dimensional integrals whose computation does not appear feasible.
k = 2: P(RZ s r) • ~ (1 0 (~n + Go}) VI 0 ~(2.1)
- 22 log (~ - 1 + 7VI - ~ ) for r ~ 4
r
k = 3:
+ a(r)64r3
1r/2
f..1r/2
(r+3-a(r)S;n(x))2cOS(X)(2b (x)(1+b2 XIr r
(2.2)
where a(r) = ((r-9)(r-l))~
and b(r) : cos(x)(cos 2(x) + 8((r-3)a(r)-2 + sin(x) a(r)-l)-~.
The trigonometric terms were introduced by a change of variable in order to
facilitate a more stable numerical quadrature. The distributions of ~2 and R3
are extremely longtailed. Simulations show that the same holds also for Rk ,
k >3. We therefore chose to tabulate the distribution function of a particular
transform of Rk• This transform was suggested by the asymptotic distribution of
Rk (Groeneboom and Pyke (1981), Groeneboom (1981)), namely
S~ - k log k L= _N(O,l)
k J3 log kas k- 00.
One easily sees however, that this convergence is entremely slow. Since Sk ? 0
one finds that the lowest value of Lk1 becomes -2(-3) when k =1.63 105 (k = 5.32
1011).
An alternate limit law is easily derived from the previous one:
Lk2 = 2(Sk - Jk log k )1 j3k LN(O,l) as k -00.
L . h b ++k2 1S muc" . e ....er aved an Lk1. For example, the lowest value
reaches -2 -3) already for k =20 k =854). However simulations for k = 50 s
that the distributions of Lk2 is still somewhat skewed with a heavy right tail.
The logarithmic transform of Sk yields the followinq limit law
Uk = 2"yl 09 k (log \ - {l09(k log k)) ~N(O,l)3
as k- 00.
We found that the distribution of Uk' k = 2, ..• , 10 is quite symmetric but still
shows somewhat heavy tails. We chose to tabulate the distribution function of Uk
partly because of the compact tables that resulted. Table 1 gives the cumulative
distribution function for U3 which was computed by adaptive Romberg quadrature.
Table 2 gives the simulated cumulative distribution functions for Uk' k =4, ..• ,
10, each based on 20,000 replications. The simulations were performed on a PDP
11/45 using the portable random number generator given by Schrage (1979).
3.0 The Power of the S2 Test
As far as the power of the Sk test is concerned, we can only offer some limited
results for k =2. In order to gain intuitive insight into the power behavior of
the S2 test , it is of interest to study the rejection region of the S2 test in
the unit square. This rejection region R is the union of the following three
sets:
where
9
f 2. u I u U
Wl th g(u) = 7 - V7+- t .
The parameter t regulates the size of the test. For t =20 the rejection region
R is depicted in Figure 1. The shape of R is certainly intriguing in that it
tends to make a special effort to reject when both P-values are moderately small.
The acceptance region is not convex which renders the $2 test inadmissible if the
P-values arise from an exponential family testing problem (Birnbaum 1959). This
negative aspect of the $2 test should however be viewed in its proper perspec
tive. The $2 test arose out of a nonparametric setting whereas the alternatives
in an exponential family testing problem appear to be rather narrow.
Figure 2 compares the rejection region of the $2 test with that of Fisher's test
for significance levels a =.1 and a = .05. We only show the part of the
rejection region that lies below the main diagonal, the other part being obtained
by reflection around it. Also note that abscissa and ordinate have different
scales. For a = .1 the boundary curves for the rejection regions cross each
other four times whereas for a = .05 there are only two crossings. Note that the
S2 test is much more prone to reject than Fisher's test in situations where one
of the P-va ues is very large and the other is small. This feature is also
brought out in the fol owing comparison. For each pair of P-values
which are to be combined, we can compute the P-value Ps of the $2 test and the D.
Actual power calculations are quite difficult in general, however for specific
simple alternatives they may be carried out. These alternatives are as follows:
PI has density 9ab(Pl) and P2 has density gcd(P2) where
with
o < u < 1, 0 s v s 1-a.
The power calculations for either test are straightforward but tedious and we
will not give the resulting formulae, but instead present some numerical results
in tables 3 and 4. Table 3 presents power results under the alternative that
both P-values have the same distribution (a =c, b = d). Table 4 presents power
results under the alternative that P1 - gab and P2 is uniform. We observe that
the S2 test outperforms Fisher's test whenever at least one of the P-values is
distributed very close to zero and Fisher's test performs better otherwise.
Acknowledgements: I would like to thank Professors Piet Groeneboom and Ron Pyke
for sharing their asymptotoic results with me and f)r. Stuart Anderson for
teaching me some things in numerical analysis.
TABLE 1
CUMULATIVE DISTRIBUTION FUNCTION OF U
FOR SAMPLE SIZE H=3
.0 .1 .2 .3 .4 .5 .6 .7 .8 .9
-6 0.00002 0.00002 0.00001 0.00001 0.00001 0.00001 0.00001 0.00001 0.00000 0.00000
-5 0.00010 0.00008 0.00007 0.00006 0.00005 0.00004 0.00004 0.00003 0.00003 0.00002
-4 0.00051 0.00043 0.00036 0.00031 0.00026 0.00022 0.00019 0.00016 0.00013 0.00011
-3 0.00273 0.00230 0.00194 0.00164 0.00139 0.00117 0.00099 0.00084 0.00071 0.00060
-2 0.01504 0.01267 0.01067 0.00899 0.00757 0.00638 0.00538 0.00454 0.00383 0.00323
-1 0.08274 0.07002 0.05917 0.04995 0.04213 0.03550 0.02991 0.02519 0.02121 0.01iB6
0 0.36424 0.32174 0.28220 0.24592 0.21308 0.18368 0.15763 0.13476 0.11483 0.09758
.0 .1 .2 .3 .4 .5 .6 .7 .8 .9
0 0.36424 0.40920 0.45600 0.50385 0.55187 0.59917 0.64487 0.68822 0.72859 0.76555
1 0.79884 0.82841 0.85433 0.87680 0.89611 0.91258 0.92653 0.93831 0.94822 0.9565:-
2 0.96350 0.96934 0.97423 0.97833 0.98176 0.98464 0.98705 0.98908 0.99079 0.99222
3 0.99343 0.99445 0.99531 0.99605 0.99666 0.99717 0.99761 0.99797 0.99828 0.99855
4 0.99877 0.99896 0.99912 0.99925 0.99937 0.99946 0.99954 0.99961 0.99967 0.99972
5 0.99976 0.99980 0.99983 0.99986 0.99988 0.99990 0.99991 0.99993 0.99994 0.99995
6 0.99995 0.99996 0.99997 0.99997 0.99998 0.99998 0.99998 0.99999 0.99999 0.99999
TABLE 2
E:1?IRICAL CL;~iUL;';IVE DISiRItUTICt{ FUHc;ION OF U
FOR SA~PLE SIZE N B~SED ON 200GO REPLICATIGHS
x
-4.9-4.8-4.7-4.6-4.5-4.4-4.3-4.2-4.1-4.0-3.9-3.8-3.7-3.6-3.5-3.4-3.3-3.2-3.1-3.0-2.9-2.8-2.7-2.6-2.5-2.4-2.3-2.2-2.1-2.0-1. 9-1.8-1.7-1.6-1.5-1.4-1.3-1.2-1.1-1.0-0.9-0.8-0.7-0.6-0.5-0.4-0.3-0.2-G.lo. a
0.00050.00050.00050.00070.000030.000030.00090.00110.00120.00120.00160.00190.00210.00230.00240.00280.00320.00370.00430.00490.00570.00650.00770.00380.01030.01170.01350.01580.01890.02250.02600.02<;80.03450.04060.04720.05390.06580.07650.OgS50.10380.12070.13840.15910.18380.2114o.24140.27180.30790.3462O.3t52
1\=5
0.00040.00040.00050.00050.00060.00070.00070.00100.00130.00150.0019o . 0-0210.00220.00230.00260.00290.00340.00380.00440.00570.00630.00710.00830.01020.01190.01420.01630.01890.02180.02570.03010.03540.04180.04870.05610.06530.07480.OS650.1005o .115S0.13390.15460.17600.20110.22960.25930.29290.32740.36560.4060
"=6
0.00050.00060.00060.00070.00090.00100.00120.00150.00160.00190.00210.00220.00260.00300.00350.00420.00480.00530.00610.00690.00790.00910.01060.01240.01440.01680.01920.02240.02630.03010.03490.03920.04530.05290.06150.07030.08150.09310.106S0.12130.13830.15960.18320.20680.23440.26270.29570.33e80.36630.4056
N=7
0.00070.00070.00090.00090.00120.00130.00160.00190.00200.00230.00270.00300.00350.00400.00430.00510.00580.00660.00790.00900.01000.01100.01310.01490.01680.02010.02280.02640.03090.03460.03950.04480.05050.05810.06730.07730.DS870.10080.11520.13060.15060.16890.1924o .21770.24620.277 90.31010.34500.38320.4217
0.00090.00100.0010O.OOlC0.00120.00140.0016O.OOB0.00230.00280.00320.00340.00400.00480.00530.00580.00660.00700.00790.00880.01020.01180.01340.01550.01820.02080.02410.02720.03050.03510.04020.04600.05310.06060.07060.07990.09050.1018o .115S0.13180.15000.16960.19100.21910.2471o . 277 00.30970.34590.37970.4158
H=9
0.0:ll10.00120.00160.001~
0.00180.00210.00240.00260.00290.00320.00360.00400.00460.00530.00580.00670.00740.00S40.00940.01080.01180.01360.01510.01730.01950.02210.02540.02SS0.03240.03690.04230.04840.05530.06300.07360.08350.09460.10800.12600.14250.1618O. IS 130.202S0.22(';70.25750.28770.31890.35190.38690.4245
H=10
0.00100.00120.00150.00160.00130.00200.00210.00250.00260.00300.00350.00390.00450.00490.00540.OC610.00680.00780.00870.00990.010130.0123O. Ul420.01570.01820.02060.02350.02670.03050.03520.04090.04710.05340.06150.Oil30.08150.09320.10600.12240.13970.15860.17890.20270.22790.25840.28850.32110.35550.39270.4292
TABLE 2 (Continued)
EMPIRICAL CUMUlATIV~ DISTRIBUTION FU~CTION OF U
FOR SAMPLE SIZE N BASED ON 20000 ~EPLICATIONS
0.1o.20.30.40.50.6o.70.80.91.01.11.21.31.41.51.61.71.81.92.02.12.22.32.42.52.62.72.82.93.03.13.23.33.43.53.63.73.83.94.04.14.24.34.44.54.64.74.84.95.0
H=4
0.4Z870.47300.51940.56310.60810.65280.69350.72970.76420.79470.82100.34610.86870.8~75
0.90470.91830.93160.94240.95190.95910.96540.97050.97630.97930.98170.98410.98610.98820.98960.99100.99190.99320.99380.99500.99560.9%10.99630.99680.99730.99730.99820.99860.99880.99390.99890.99920.99930.99940.9'1950.9996
1'1=5
0.44630.48730.52920.57350.61310.65110.68970.72390.75810.78S70.81560.83850.S6160.8S120.89340.91l80.92440.93570.94570.95420.96050.96570.97020.97440.97800.98060.9836o. 985&0.98740.98930.9909O. 99220.99370.99460.99520.99570.99620.99650.99720.99740.99760.99790.99810.99320.99&50.99870.99380.99910.99920.9993
0.44710.48810.53060.56980.60890.64S40.68770.72360.7563o.13600.S1l70.83850.85920.87810.89370.90710.91980.93090.94020.94810.95490.96000.96560.97070.97420.97720.98030.98270.98460.98640.98800.98950.9905'0.99170.99260.99370.99440.99520.99580.99640.99670.99680.99730.99770,99790.99310.99830.99840.~985
0.9987
1'1=7
0.46100.50000.54090.58150.62060.65680.69240.72600.75900.78930.81530.84040.85970.87790.89500.90920.92190.93250.94130.94900.95660.96250.96760.97180.9752o. 97840.98110.98390.98560.98740.98860.99010.99150.99240.99320.99420.99470.99550.99620.99660.99700.99720.99740.99790.99810.99520.9(;830.99~6
0.9'1S90.9,t.9
0.45550.49590.53480.57470.b1430.65120.68910.72310.75650.78440.81020.8351o.85690.87740.89410.90720.91S70.92980.93930.94680.95390.95960.96500.96890.97250.97590.97910.98160.98400.98590.98750.98950.99050.99170.99260.99340.99420.99470.99540.99600.99640.99670.99700.99740.99770.9979O.99Z20.9983O.9C;S50.9936
0.46370.50280.54550.58300.62170.65750.69310.72570.75710.18570.81230.83550.85700.87500.89210.90550.91650.92660.93570.94430.95060.95700.96260.96720.97140.97450.97760.98050.98310.98500.98690.98850.98990.99100.99180.99260.99340.99420.99470.99530.99580.99650.99700.99730,99770.9979O.9C;S10.<;9S30.99870.9953
H=10
0.46730.50670.54310.58280.62290.66080.69460.72810.75990.78780.81250.83780.85620.87560.89160.90500.91830.92920.93750.94590.95260.95930.96360.96780.97200.97510.97770.98040.98260.98470.9863o. 9S 770.98890.9904O.99~5
0.99250.99320.9940O.9<,l460.99530.99560.99600.99650.9';72o. 9 <17:·0.99790.9981a.9985C.C;gS7O.9C;90
b .05
a = .02 F .1248
$2 .1445
a = .05 F .0954
$2 .0987
a = .08 F .0837
$2 .0839
a = .10 F .0791
$2 .0778
a = .20 F .0672
$2 .0640
a = .30 F .0622
$2 .0593
a = .05
.1
.1968
.2340
.1412
.1475
.1194
.1198
.1109
.1076
.0862
.0785
.0756
.0689
TABLE 3
a = c
.2
.3323
.3982
.2345
.2456
.1970
.1977
.1827
.1727
.1300
.1094
.1056
.0889
b = d
.3
.4565
.5426
.3299
.3443
.2829
.2839
.2655
.2454
.1812
.1426
.1401
.1101
.5
.6708
.7721
.5268
.5434
.4796
.4806
.4637
.4136
.3060
.2160
.2225
.1558
.8
.9072
.9680
.8376
.8465
.8364
.8369
.8430
.7225
.5495
.3436
.9
.9633
.9937
.9454
.9487
.9719
.9720
.9912
.8407
Power of Fisher's (F) test and the S2 test when both P-val ues have the same
density gab x).
15
TABLE 4
a = .05 d = a
S2 .0985
S2 .0743
S2 .0664
S2 .0634
$2 .0568
$2 . 0546
b
a = .02 F
a = •as F
a = .08 F
a = .10 F
a = .20 F
a = .30 F
.05
.0881
.0726
.0663
.0639
.0581
.0558
.1
.1263
.1469
.0951
.0986
.0826
.0829
.0777
.0769
.0662·
.0637
.0617
.0592
.2
.2025
.2439
.1402
.1472
.1153
.1157
.1055
.1038
.0825
. 0773 .
.0734
.0683
.3
.2788
.3408
.1853
.1958
.1479
.1486
.1332
.1307
.0987
.0910
.0850
.0775
.5
.4313
.5347
.2755
.2930
.2131
.2143
.1886
.1845
.1312
.1184
.1084
.0959
.8
.6600
.8255
.4108
.4388
.3110
.3128
.2718
.2651
.1800
.1594
.9
.7363
.9224
.4559
.4875
.3436
.3457
.2996
.2920
Power of Fis 's F test the S2 test when PI has
uniformly distributed.
a , V
.8
.6
. 4
.2
-
.
-liZ
. --III
I R)II I I
o .2 .4 .6 .8 1.0
Figure 1. Rejection Region of the $2 Test
• It.
--- ----- Fisher's test
--- 52 test
- -- - .......
//
//
I'a • .1/ ',,(
/~t12"""" ----a • .05 • - - - - - - - - - - - _ _ _ - - - . - --------- ------ .....
.28
.20
.24
.16
.08
.12
.04
•." . 2 .3 .4 .5 .6 .7 .8 .9 1.0
Figure 2. Comparison of Rejection Regions
1.0~
~,,.9 ,
Ps < PF~,
~,~
.8 ,,,,,.7 PF < Ps
,~
i,,,
Ps < PF,
.6,,,,,
~
.5 ,
,.4
,,,,
,.3 ,
.2PF < Ps
.1 ,Ps < PF,,,
0,
.1 .2 .3 .4 .5 .6 .7 .8 .9 1.0
Figure 3. Comparison of Ps and PF
References
Barlow, R.E., Bartholomew, D.J., Bremner, J.M. and Brunk, H.O. (1972),Statistical Inference Under Order Restrictions, John Wiley &Sons, London, ~ew
York.
Behnen, K. (1975) "The Randles-Hogg Test and an Alternative Proposal," Corrrn. inStat. 4(3), 203-238.
Birnbaum, A. (1954), "Combining Independent Tests of Significance," J. AmericanStatist. Assoc., 49, 559-575.
Fisher, R.A. (1932), Statistical Methods for Research Workers, Oliver and Boyd,Edinburgh, London, 4th ed.
Groeneboom P. and Pyke R. (1981), "Asymptotic Normality of Statistics Based onConvex Minorants of Empirical Distribution Functions," Technical Report No.5,Department of Statistics, University of Washington, Seattle.
Groeneboom, P. (1981), "The Concave Majorant of Brownian Motion," TechnicalReport No.6, Department of Statistics, University of Washington, Seattle.
KOZiol, J.A. and Perlman, M.D. (1978), "Combining Independent Chi-Squared
Tests," J. American Statist. Assoc., 73, 753-763.
Lehmann, LL. (1959), Testing Statistical Hypotheses, John Wiley & Sons, New
York, London.
liptak, T. (1958), "On the Combination of Independent Tests," Magyar Tud. Akad.
Mat. Kutato Int. Kozl. 3, 171-197.
Mudholkar, G. S. and George, E. O. 1979, "The Logit Statistic for Comb inProbabilities - an Overview," OptimiZing Methods in Statistics, Rustagi
Academic Press.
9
Oosterhoff, J. (1969), Combination of One-~ided Statistical Tests, MathematischCentrum, Amsterdam.
Pyke, R. (1965), "Spacings," J. R. ~tatist. Soc. B 27, 395-436.
Roy, S. N. (1953), "On a Heuristic Method of Test Construction and its Use inMultivariate Analysis," Ann. Math. Stat. 24, 220-238.
Schrage, L. (1979), "A More Portable Fortran Random Number Generator," ArM Trans.
Math. Software, S, 132-138.
Tippet, L. H. C. (1931), The Method of Statistics, First edition, Williams and
Norgate, London.