Randomicity, Predicitability, and Mathematical Inference Strategies in ESP … · 2013-08-08 ·...
40
dRo& . - Randomicity, Predictability, and Mathematical Inference Strategies 5 1 in ESP Feedback Training Situations Charles T. Tart Psychology Department University of California - 5% D a v i s , CA 95616 7- 6 7- Y, I have long believed that the most pressing problem in parapsychology is how to get strong and reliable manifestations of psi in the lab, so we can profitably get on to the important questions of what psi is and how it works. As a result, some years ago I rather innocently set out to de- termine if providing immediate feedback of results to percipients would allow them to at least stabilize, if not actually strengthen, their ESP abilities. Little did I realize what a hornets' nest of controversy I would stir up! The initial research, published in my Parapsychology Foundation monograph (Tart, 1975), and more widely in my Learning -- to Use Extrasensory Perception book (Tart, 1976a), has stood up rather well to lengthy questionings and attacks from O'Brien (1976), Stanford (1977), Gardner (1977), but today I find myself under fire from Gatlin's guns. I shall try to address myself to some useful questions that arise when im- mediate feedback of results is provided to percipients. When the vast majority of ESP studies were done without immediate feed- ! back to the percipients, the question of possible biases in target sequences, significant departures from equiprobability and serial independence, arti- factually inflating the results was rather easily handled. Unless the global biases of the percipients just happened to match the global biases of the target sequence, a matter that could be checked by control matchings, such biases were not important unless of very large magnitude. With the increas- ing use of immediate feedback about target identity, in an effort to stabilize
Transcript of Randomicity, Predicitability, and Mathematical Inference Strategies in ESP … · 2013-08-08 ·...
dRo& .-
Randomicity, P r e d i c t a b i l i t y , and Mathemat ical I n f e r e n c e S t r a t e g i e s 5 1
i n ESP Feedback T r a i n i n g S i t u a t i o n s
Char les T. T a r t
Psychology Department
U n i v e r s i t y o f C a l i f o r n i a - 5%
Davis, CA 95616 7- 6 7- Y,
I have long b e l i e v e d t h a t t h e most p r e s s i n g problem i n parapsychology
i s how t o g e t s t r o n g and r e l i a b l e m a n i f e s t a t i o n s of p s i i n t h e l a b , s o we
can p r o f i t a b l y g e t on t o t h e impor tan t q u e s t i o n s of what p s i i s and how
i t works. A s a r e s u l t , some y e a r s ago I r a t h e r i n n o c e n t l y s e t o u t t o de-
t e rmine i f p r o v i d i n g immediate feedback of r e s u l t s t o p e r c i p i e n t s would
a l l o w them t o a t l e a s t s t a b i l i z e , i f not a c t u a l l y s t r e n g t h e n , t h e i r ESP
a b i l i t i e s . L i t t l e d i d I r e a l i z e what a h o r n e t s ' n e s t o f c o n t r o v e r s y I
would s t i r up! The i n i t i a l r e s e a r c h , pub l i shed i n my Parapsychology
Foundat ion monograph ( T a r t , 1975) , and more wide ly i n my L e a r n i n g -- t o Use
E x t r a s e n s o r y P e r c e p t i o n book ( T a r t , 1976a) , has s t o o d up r a t h e r w e l l t o
l e n g t h y q u e s t i o n i n g s and a t t a c k s from O'Brien (1976), S t a n f o r d (1977) ,
Gardner (1977) , b u t today I f i n d myself under f i r e from G a t l i n ' s guns. I
s h a l l t r y t o a d d r e s s myself t o some u s e f u l q u e s t i o n s t h a t a r i s e when im-
media te feedback of r e s u l t s i s provided t o p e r c i p i e n t s .
When the v a s t m a j o r i t y o f ESP s t u d i e s were done w i t h o u t immediate feed- !
back t o t h e p e r c i p i e n t s , t h e q u e s t i o n o f p o s s i b l e b i a s e s i n t a r g e t sequences ,
s i g n i f i c a n t d e p a r t u r e s from e q u i p r o b a b i l i t y and s e r i a l independence, a r t i -
f a c t u a l l y i n f l a t i n g t h e r e s u l t s was r a t h e r e a s i l y handled. Unless t h e g l o b a l
b i a s e s o f t h e p e r c i p i e n t s j u s t happened t o match t h e g l o b a l b i a s e s o f t h e
t a r g e t sequence, a m a t t e r t h a t cou ld be checked by c o n t r o l match ings , such
b i a s e s were n o t impor tan t u n l e s s o f v e r y l a r g e magnitude. With t h e i n c r e a s -
i n g use o f immediate feedback abou t t a r g e t i d e n t i t y , i n a n e f f o r t t o s t a b i l i z e
T a r t -2
and improve ESP performance ( T a r t , 1966 ; 1977e) , q u e s t i o n s abou t p o s s i b l e
e f f e c t s o f b i a s e s i n t h e t a r g e t sequences a r e more i m p o r t a n t , f o r we can con-
c e i v e of a p e r c i p i e n t g r a d u a l l y l e a r n i n g what t h e b i a s e s of the t a r g e t se -
quences a r e and then a l t e r i n g h i s own response s t r a t e g i e s t o t a k e advantage
o f them, t h u s c r e a t i n g a r t i f a c t u a l " h i t s " which might t e l l us something
abou t mathemat ica l i n f e r e n c e s t r a t e g i e s , , b u t l i t t l e o r n o t h i n g about ESP.
I n t h e b r i e f t ime a l l o t t e d t o me I s h a l l t r y t o o u t l i n e some new per-
s p e c t i v e s on t h i s i s s u e t h a t I have developed, such a s f i n d i n g t h a t t h e
s t a n d a r d Chi-square measures of b i a s . a r e n o t v e r y u s e f u l measures of the
p r e d i c t a b i l i t y of a b i a s e d sequence, and p r e s e n t a b r i e f d e s c r i p t i o n of the
r e s u l t s o f a powerful mathemat ical i n f e r e n c e p r e d i c t o r program developed by
Eugene Dronek and m e t h a t a t t a c k s t h e problem of p r e d i c t i o n of b i a s e d sequences
d i r e c t l y . U n f o r t u n a t e l y , whi le i t i s u s e f u l t h a t D r . G a t l i n r a i s e d t h i s
i s s u e here today, he r own s o l u t i o n t o i t i s g r a v e l y flawed i n a v a r i e t y o f T' Wtl , be q,t u4, /ee<G& &&- &d'/ bi $ey,.zj-&IC f f ics I ways, and t h e p a r t i c u l a r c o n c l u s i o n s she ha reached a r e i n v a l i d . These f l a w s
?=i
i n c l u d e such t h i n g s a s a p e r s i s t e n t f a i l u r e t o unders tand t h e - d i f f e r e n c e be-
tween v a l i d p r e d i c t i o n and t r i v i a l p o s t d i c t i o n , t h e c l a s s i c a l e r r o r of equa t -
i n g c o r r e l a t i o n w i t h c a u s a t i o n , c o n f u s i n g p o t e n t i a l and p r o o f , c l a i m i n g en-
hanced s e n s i t i v i t y f o r her s t a t i s t i c a l p rocedures when they p robab ly l a c k
. v a l i d i t y , making a t l e a s t one major c l a i m f o r which she p r e s e n t s no s u p p o r t i n g
d a t a a t a l l , and i n t e r p r e t i n g i n v a l i d s t a t i s t i c a l a b s t r a c t i o n s i n ways which !
would have r e v e a l e d themselves a s o b v i o u s l y f a l s e i f s h e had looked a t t h e
raw d a t a they were based on. I s h a l l d e t a i l t h e s e problems below, a s they
a p p l y t o D r . ~ a t l i n ' s a n a l y s e s of my feedback t r a i n i n g d a t a . D r . P r a t t w i l l
comment on D r . G a t l i n ' s a n a l y s i s of t h e M a r t i n - S t r i b i c d a t a .
G a t l i n ' s Main A s s e r t i o n s :
A s a number o f c o l l e a g u e s have remarked t o me t h a t they have had d i f f i -
c u l t y f o l l o w i n g t h e w r i t t e n v e r s i o n o f D r . ~ a t l i n ' s paper ( G a t l i n , 1 9 7 8 ~ ) ~
T a r t -3
l e t me b r i e f l y summarize h e r main a s s e r t i o n s . She a s s e r t s t h a t :
( 1 ) . Given two n u m e r i c a l s e q u e n c e s t h a t n o t o n l y a r e e a c h b i a s e d , b u t
have ma tch ing p a t t e r n s o f b i a s e s , when you compare t h e s e s e q u e n c e s w i t h e a c h
o t h e r you w i l l g e t a h i g h e r number o f i d e n t i c a l numbers i n t h e same p o s i t i o n s ,
h i t s , t h a n you would e x p e c t i f you m i s t a k e n l y assume t h a t t h e s e q u e n c e s a r e
random and unb iased .
( 2 ) . Both some t a r g e t and many r e s p o n s e s e q u e n c e s i n my f i r s t f e e d b a c k
T r a i n i n g S t u d y show s i g n i f i c a n t d e g r e e s o f b i a s , a t t h e s i n g l e t , d o u b l e t , and
t r i p l e t l e v e l s .
(3) . Because o f immedia te f e e d b a c k a b o u t t h e i d e n t i t y o f e a c h t a r g e t ,
d a t a is a v a i l a b l e t o p e r c i p i e n t s from which t h e y migh t c a l c u l a t e c h a r a c t e r i s -
t i c s o f p o s s i b l e b i a s e s i n t h e i r i n d i v i d u a l t a r g e t s equences .
(4). The human mind has " f a n t a s t i c " ( h e r term) c a p a b i l i t i e s , presumably
u n c o n s c i o u s , f o r p a t t e r n r e c o g n i t i o n i n n u m e r i c a l s e q u e n c e s t h a t a l l o w u s e
o f t h e i n f o r m a t i o n o b t a i n e d t h r o u g h feedback : a s D r . G a t l i n p u t s i t , ' I . . . e x t r e m e l y s u b t l e b i a s e s a t h i g h n - t u p l e l e v e l s i n f i n i t e s e q u e n c e s c a n be
u t i l i z e d by t h e human mind."
( 5 ) . The p e r c i p i e n t s i n my s t u d y n o t o n l y had t h e p o t e n t i a l t o u t i l i z e
s u c h b i a s p a t t e r n s , t h e y did u s e them t o o b t a i n mos t o r a l l o f t h e h i t s c o r e s
above chance e x p e c t a t i o n . T h e r e f o r e , D r . G a t l i n asserts t h a t :
( 6 ) . A l l o f t h e above-chance s c o r i n g i n my f i r s t T r a i n i n g S t u d y c a n be !
e x p l a i n e d by p e r c i p i e n t s f i g u r i n g o u t and u t i l i z i n g t h e t a r g e t s equence b i a s e s
w i t h a n unconsc ious m a t h e m a t i c a l i n f e r e n c e s t r a t e g y , s o t h e r e i s no need t o
p o s t u l a t e ESP a s a n e x p l a n a t i o n o f t h e d a t a .
The re a r e o t h e r m i s c a l l a n e o u s a s s e r t i o n s i n D r . ~ a t l i n ' s p a p e r , b u t I
b e l i e v e I have a d e q u a t e l y o u t l i n e d he r main argument h e r e .
What Are t h e B i a s e s i n t h e T a r g e t Sequences?
Al though I am p r o b a b l y more f a m i l i a r w i t h D r . ~ a t l i n ' s D-measures o f b i a s
T a r t -4
t h a n most of you, I s t i l l f i n d them d i f f i c u l t t o f o l l o w , s o i n o r d e r t o look
a t p o t e n t i a l b i a s e s * i n t h e t a r g e t sequences of my f i r s t T r a i n i n g S tudy I
s h a l l pres 'ent them i n more f a m i l i a r Chi-square n e a s u r e s , t o which G a t l i n t s
** D-measures a r e s o s t r o n g l y r e l a t e d t h a t f o r p r a c t i c a l purposes t h e y a r e
e q u i v a l e n t , i n s p i t e o f he r s t r e s s on t h e i r uniqueness . Table 1 p r e s e n t s
b o t h Chi s q u a r e measures I have computed .and t h e few D-measures D r . G a t l i n
p r e s e n t e d i n her paper ( s h e p r e s e n t e d o n l y those r e a c h i n g s t a t i s t i c a l s i g n i -
f i c a n c e ) f o r t h e s i n g l e t ' and d o u b l e t l e v e l s . ,~. -,
, , , -, I_ I I I I > I I Y W - . . _ ... . - - - - - - - - - - - - - - - - - - -
I n s e r t Table 1 about he re __ .,..
Two n o t e s on t h e v a l u e s i n Table 1 should be made. My Chi-squares (and
o t h e r c a l c u l a t i o n s ) w i l l be c l o s e t o b u t sometimes n o t e q u i v a l e n t t o any c a l -
c u l a t e d from D r . ~ a t l i n ' s a n a l y s e s , a s she t r e a t e d t h e d a t a I provided her
i n a s l i g h t l y less a c c u r a t e manner by f i l l i n g i n t a r g e t d a t a a s s o c i a t e d w i t h
Passes (no response) by t h e p e r c i p i e n t s w i t h a response d i g i t from her computer
*I s h a l l use t h e term "bias" i n a g e n e r a l way i n t h i s paper t o d e s c r i b e even
t h e s l i g h t e s t d e v i a t i o n from a n e q u i p r o b a b i l i t y and s e r i a l independence model,
w i t h t h e q u e s t i o n o f whether such b i a s i s o n l y a random f l u c t u a t i o n o r is
r s t a t i s t i c a l l y o r p r a c t i c a l l y s i g n i f i c a n t handled s e p a r a t e l y .
**I c o r r e l a t e d f o r s e t s o f s i n g l e t and d o u b l e t D-measures and Chi-squares
andy)-chi-squares (Davis & Akers, 1974), c a l c u l a t e d by D r . G a t l i n , working
from a computer and p r i n t o u t s h e p rov ided me w h i l e s h e was s t i l l honor ing her
commitment t o p r o v i d e me w i t h c o p i e s o f a l l a n a l y s e s she c a r r i e d o u t on my
d a t a , a n d found t h e c o r r e l a t i o n s t o range from .91 t o 1.00, f o r an average
1 c o r r e l a t i o n o f .97.
T a r t -5
,'s pseudorrandom number g e n e r a t o r program, whereas I d e l e t e d t h e s e t r i a l s , s i n c e
t h e p e r c i p i e n t s d i d n o t r e c e i v e feedback when t h e y passed. Second, t h e Chi-
s q u a r e v a l u e s I have c a l c u l a t e d f o r t h e s i n g l e t l e v e l i n Table 1 a r e based
on a model of e q u i p r o b a b i l i t y o f a l l s i n g l e t s (p = . l o ) , b u t s i n c e t h e r e i s
some s i n g l e t b i a s , t h e d o u b l e t l e v e l Chi-square c a l c u l a t i o n s a r e c o r r e c t e d
f o r s i n g l e t b i a s by b e i n g based on m a r g i n a l t o t a l s , r a t h e r t h a n t h e o r e t i c a l
v a l u e s . Without t h i s c o r r e c t i o n , s i g n i f i c a n t Chi-square v a l u e s a t t h e doub-
l e t l e v e l migh t be o n l y r e f l e c t i o n s o f s i n g l e t b i a s , r a t h e r t h a n v a l i d l y i n d i -
c a t i n g a h i g h e r o r d e r b i a s . I n s o f a r a s I unders tand D r . G a t l i n ' s D;(T) mea-
s u r e i t a l s o c a l c u l a t e s d o u b l e t b i a s independent o f s i n g l e t b i a s .
The f a c t shown i n Tab le 1, t h a t seven o f D r . ~ a t l i n ' s d o u b l e t l e v e l
b i a s measures a r e s i g n i f i c a n t , when o n l y t h r e e o f mine a r e , i s a n i n t e r e s t -
i n g d i s c r e p a n c y . D r . G a t l i n c l a i m s g r e a t e r " s e n s i t i v i t y " f o r her D-measures
t h a n f o r c o n v e n t i o n a l Chi-square measures . Whether t h i s c l a i m o f s e n s i t i v i t y
o f h e r D-measures i s a c t u a l l y v a l i d , o r j u s t r e p r e s e n t s a n a r b i t r a r y lower ing
o f s t a n d a r d s f o r s i g n i f i c a n c e i s a p o i n t I w i l l l e a v e f o r t h e more s t a t i s t i -
c a l l y e r u d i t e t o work o u t , b u t what we shou ld n o t e h e r e i s t h a t t h e " s i g n i f i c a n t "
d e p a r t u r e s from t h e model t h a t D r . G a t l i n c l a ims t o have d e t e c t e d w i t h he r
D-measures a r e even t i n i e r t h a n t h o s e d e t e c t e d w i t h t h e Chi-square and, a s
w e s h a l l see l a t e r , t i n y d e p a r t u r e s from e q u i p r o b a b i l i t y and s e r i a l independence
!
* ~ r . G a t l i n d e s c r i b e s u s i n g a random number g e n e r a t o r program i n he r paper ,
b u t t h e computer a t UC Berke ley where s h e c a r r i e d o u t he r a n a l y s e s u s e s a
pseudo-random g e n e r a t o r w i t h a n a lgorhythm, a s p r a c t i c a l l y a l l computer ran-
dom g e n e r a t o r programs do. I t i s p r o b a b l y s a t i s f a c t o r i l y random f o r s h o r t
sequences .
T a r t -6
may n o t be p r a c t i c a l l y u s e f u l f o r making h i t s w i t h a m a t h e m a t i c a l i n f e r e n c e
s t r a t e g y .
As t h e Ch i - squa re measures i n T a b l e 1 show, two t a r g e t s e q u e n c e s were
s i g n i f i c a n t l y b i a s e d a t t h e s i n g l e t l e v e l and t h r e e a t t h e d o u b l e t l e v e l .
TWO o+ J" J-5, P 1 ~ - i x ~ ( ~ l , erst b i w - - -C - Why migh t t h i s have o c c u r r e d ?
I n r e p o r t i n g on t h i s b i a s i n . e a r l i e r p u b l i c a t i o n s ( T a r t , 1977a; 1977b) ,
I p o i n t e d o u t t h a t p r i o r t o c o l l e c t i n g d a t a i n t h e f i r s t s t u d y , my c o l l e a g u e s
and I were aware o f t h e many s t u d i e s which showed t h a t s u b j e c t s ' d e s i r e s t o
a l t e r t h e o u t p u t c o u l d s i g n i f i c a n t l y ' a f f e c t e l e c t r o n i c random number genera-
t o r s (RNGS). A l though we wanted o u r p e r c i p i e n t s t o b e o n l y p e r c i p i e n t s and
n o t a g e n t s , i .e . , t o u s e ESP b u t n o t PK, t h e y n e v e r t h e l e s s c o u l d s c o r e w e l l
b y u n c o n s c i o u s l y PKing o u r e l e c t r o n i c RNG t o make i t s o u t p u t f i t t h e i r r e s p o n s e
p r e f e r e n c e s . Our i n s t r u c t i o n s t o t h e p e r c i p i e n t s t o t r y a v a r i e t y o f s t ra-
t e g i e s may have f u r t h e r enhanced t h e PK p o s s i b i l i t y . For t h i s r e a s o n , we
made a d e c i s i o n b e f o r e s t a r t i n g t h e e x p e r i m e n t t o l e t t h e s a t i s f a c t o r i n e s s
o f o u r RNG rest on two samples o f 1 ,000 t a r g e t s e a c h , t a k e n b e f o r e we i n t r o -
duced our p e r c i p i e n t s t o t h e equipment and a f t e r t h e l a s t p e r c i p i e n t had
f i n i s h e d t h e s t u d y . These checks showed s a t i s f a c t o r y r a n d o m i c i t y a t t h e s i n g -
l e t and d o u b l e t l e v e l s . D r . G a t l i n h e r s e l f r e p o r t s ( G a t l i n , 1978b) t h a t t h e
e n t i r e t a r g e t s e q u e n c e f o r t h e T r a i n i n g S t u d y (5 ,000 t r i a l s ) shows s a t i s f a c -
! t o r y r a n d o m i c i t y a t t h e s i n g l e t and d o u b l e t l e v e l s . The f i n d i n g o f s i g n i f i -
c a n t non-randomici ty i n some i n d i v i d u a l t a r g e t s e q u e n c e s is t h u s n o t unexpec ted ,
a l t h o u g h t h e c u r r e n t i s s u e s would n o t have a r i s e n i f a l l t h e t a r g e t s equences
had shown r a n d o m i c i t y by t h e s t a n d a r d Chi-square tests. I n r e t r o s p e c t , t h e
a p p e a r a n c e o f b i a s i n some s e q u e n c e s h a s been a n a d v a n t a g e , s e r v i n g a s a s t i m -
u l u s t o c l a r i f y some i m p o r t a n t i s s u e s .
An o b s e r v a t i o n s u p p o r t i n g t h e i d e a o f PK a s t h e r e s p o n s i b l e f a c t o r f o r
t h e s i n g l e t b i a s e s i s t h a t t h e h i g h number i n t h e two s i g n i f i c a n t s equences
T a r t -7
i s n o t t h e same, a f i n d i n g s e n s i b l e i n l i g h t o f p s y c h o l o g i c a l number p r e f e r -
e n c e s which would p robab ly d i f f e r between i n d i v i d u a l s , b u t which would n o t
seem l i k e l y t o a r i s e f o r e l e c t r o n i c r e a s o n s .
My s u g g e s t i o n o f PK a s a p o s s i b l e e x p l a n a t i o n f o r l a c k o f randomic i ty
i n two o f t h e t e n t a r g e t sequences i s concerned main ly w i t h t h e s i n g l e t l e v e l
o f b i a s , a s t h e r e i s a more p r o s a i c e x p l a n a t i o n f o r t h e t h r e e t a r g e t sequences
which show s i g n i f i c a n t d o u b l e t b i a s . As e x p l a i n e d e l s e w h e r e ( T a r t , 1977b),
t h e RNG was b u i l t w i t h a pushbut ton s w i t c h t o a c t i v a t e i t f o r e a c h t r i a l .
Th i s pushbut ton was n o t o f t h e type t h a t makes a d i s c e r n i b l e "snap" o r " c l i c k "
when i t i s d e p r e s s e d , b u t a cheaper type i n which r e s i s t a n c e t o b e i n g pushed
s t e a d i l y i n c r e a s e s w i t h b u t t o n t r a v e l . I n i n t e r v i e w i n g some e x p e r i m e n t e r s
a f t e r t h e s i g n i f i c a n t d o u b l e t b i a s was found, t h e y p o i n t e d o u t t h a t sometimes
they would push t h e pushbut ton t o o b t a i n t h e n e x t t a r g e t number, n o t i c e t h a t
t h e number on t h e e l e c t r o n i c d i s p l a y had n o t changed, and then assume t h a t
they had n o t pushed t h e b u t t o n hard enough t o make c o n t a c t and a c t i v a t e t h e
RNG, s o t h e y would push i t again! T h i s would produce a g r e a t d e f i c i e n c y o f
XX d o u b l e t s ( l , l s , 2 , 2 s , e t c . ) , and , indeed , t h i s l a c k of XX d o u b l e t s is the
major c o n t r i b u t i o n t o t h e s i g n i f i c a n t d o u b l e t r e s u l t s . I f t h e t e n XX d o u b l e t
c e l l s a r e l e f t o u t o f t h e Chi-square c a l c u l a t i o n s , two o f the t h r e e s i g n i f i -
c a n t d o u b l e t t e s t s f a l l t o i n s i g n i f i c a n c e and t h e t h i r d one is j u s t s i g n i f i -
c a n t a t t h e .05 l e v e l . r ; ~ 1 1 ~
\fl I n summary, we have two o f t e n t a r g e t sequences which a r e s i g n i f i c a n t l y
be &If C ~ U I C L P vkiaG* , A ~ i l t h o ~ ~ h &> b i a s e d a t t h e s i n g l e t l e v e l , one t h a t i s s i g n i f i c a n t l y b i a s e d a t the d o u b l e t
A l e v e l independent of t h e XX d o u b l e t l a c k , and two t h a t a r e s i g n i f i c a n t l y b i a s e d
a t t h e d o u b l e t l e v e l due t o t h e exper imente r e r r o r which s y s t e m a t i c a l l y de-
p l e t e d XX d o u b l e t s from t h e t a r g e t sequences . The p o s s i b i l i t y o f i n f l a t e d
s c o r i n g through matching o f t h i s l a c k o f XX d o u b l e t s w i t h s i m i l a r b i a s e s on
t h e p e r c i p i e n t s ' p a r t has a l r e a d y been d i s c u s s e d i n t h e l i t e r a t u r e ( T a r t , 1977a;
T a r t -8
1977b; 1 9 7 7 f ) , and shown t o be t r i v i a l , g i v e n t h e v e r y h igh l e v e l of s c o r i n g
i n t h e s tudy .
T r i p l e t Leve l B i a s e s ?
What a r e consp icuous by t h e i r absence i n Table 1 a r e any t r i . p l e t l e v e l
t e s t s , a l t h o u g h D r . G a t l i n r e p o r t s h e r D; (T) measures and makes a number o f
i n t e r p r e t i v e s t a t e m e n t s a b o u t t h e i r s i g n i f i c a n c e . Unless he r D; measure has
v a s t l y d i f f e r e n t p r o p e r t i e s t h a n a t r i p l e t l e v e l Chi-square , however, i t seems , f i g u r e s
c e r t a i n t h a t her are i n v a l i d . I n computing any s t a t i s t i c , we need a c e r -
t a i n minimum sample s i z e i n o r d e r t o assume i t i s r e a s o n a b l y r e p r e s e n t a t i v e
of t h e p o p u l a t i o n i t is drawn from. For Chi-square t e s t s , t h i s i s u s u a l l y
e x p r e s s e d a s t h e r u l e t h a t t h e e x p e c t e d v a l u e i n e a c h c e l l o f t h e m a t r i x
must be f i v e o r g r e a t e r .
I n do ing he r t r i p l e t l e v e l t e s t s , D r . G a t l i n i s s p r e a d i n g a mere 500
d a t a p o i n t s over 1 , 0 0 0 c e l l s , f o r an e x p e c t e d v a l u e of o n l y one-hal f i n e a c h ,
v i o l a t i n g t h e minimum e x p e c t a t i o n r u l e by a f a c t o r of 10. Simple c a l c u l a t i o n
w i l l show t h a t v i o l a t i o n o f t h i s r u l e l e a d s t o g r o s s l y i n f l a t e d Chi-square
v a l u e s . For example, i f e v e r y t r i p l e t i n t h i s sample o f 500 were e x a c t l y
e q u i p r o b a b l e , t h a t i s t h a t t h e y were d i s t r i b u t e d over 500 s e p a r a t e c e l l s
i n t h e t r i p l e t t a b l e , c a l c u l a t i o n would g i v e a s u p e r - s i g n i f i c a n t Chi-square ,
c o r r e s p o n d i n g t o a CR o f -221 D r . G a t l i n e r r s i n even p r e s e n t i n g such in-
! v a l i d measures , much less i n t e r p r e t i n g what s h e t h i n k s t h e y show abou t t h e
p e r c i p i e n t s ' r e s p o n s e p a t t e r n s , o r i n t e r p r e t i n g them a s e v i d e n c e f o r t h e
e x i s t e n c e of h i g h e r o r d e r b i a s p a t t e r n s i n t h e t a r g e t sequences . Indeed , D r .
G a t l i n must n o t have b o t h e r e d t o compare he r c o n c l u s i o n s abou t t h e s e i n v a l i d
D; a b s t r a c t i o n s w i t h t h e a c t u a l d i s t r i b u t i o n of p a t t e r n s i n t h e d a t a a v a i l -
a b l e t o h e r , o r s h e c o u l d n o t have made t h e i n t e r p r e t i v e s t a t e m e n t s s h e made
a b o u t them. Much o f he r c l a i m abou t t h e a l l e g e d s u p e r i o r s e n s i t i v i t y o f he r
T a r t -9
D-measures then , r e s t s on i n v a l i d a n a l y s i s p rocedures . I s h a l l d r o p a l l
f u r t h e r r e f e r e n c e t o D r . ~ a t l i n ' s t r i p l e t d a t a .
D r . G a t l i n r e p o r t s t h a t t h e s c o r i n g r a t e o f t h e p e r c i p i e n t s i s s i g n i -
f i c a n t l y c o r r e l a t e d w i t h t h e d e g r e e o f d e p a r t u r e from a n e q u i p r o b a b i l i t y
and s e r i a l independence model. Th i s i s t r u e . U n f o r t u n a t e l y , s h e goes on t o
make t h e c l a s s i c a l s t u d e n t e r r o r of e q u a t i n g c o r r e l a t i o n w i t h c a u s a t i o n when
she s t a t e s "The s c o r i n g r a t e i s s i g n i f i c a n t l y p o s i t i v e l y c o r r e l a t e d w i t h
1 1 D l (T) and D2 (T) . . . . . . C l e a r l y t h e p a t t e r n i n g i n t h e t a r g e t i s b e i n g
used b~ (my i t a l i c s ) t h e s u b j e c t s t o i n f l a t e t h e i r s c o r e s . " ( G a t l i n , 1978c, - P 1 -
I a g a i n s u s p e c t o c c a s i o n a l PK by t h e p e r c i p i e n t s a s t h e cause of t h i s
c o r r e l a t i o n , w i t h t h e more s u c c e s s f u l p e r c i p i e n t s o c c a s i o n a l l y (unconsc ious ly )
t r y i n g a PK s t r a t e g y i n a d d i t i o n t o t h e i r ESP s t r a t e g i e s . Whatever t h e
c a u s e o f t h i s c o r r e l a t i o n , t h e c o n s i d e r a t i o n s I s h a l l now d i s c u s s e s t a b l i s h
t h a t t h e e x i s t e n c e o f s i g n i f i c a n t (and c o n s i s t e n t ) b i a s p a t t e r n s i n a t a r g e t
sequence e s t a b l i s h e s o n l y a p o t e n t i a l f o r u s i n g a mathemat ica l e s t i m a t o r s t r a -
t e g y , a p o t e n t i a l t h a t may n o t b e p r a c t i c a l l y u s e f u l . I t i s impor tan t t o
n o t e t h a t t h i s p o t e n t i a l e x i s t e d i n o n l y two o f t h e t e n t a r g e t sequences ( f o r
P3 and P5) of t h e f i r s t T r a i n i n g S t u d y , and a s imple and t r a d i t i o n a l way t o
handle i t would be t o j u s t d e l e t e t h e d a t a from those two p e r c i p i e n t s . T h i s
! would s t i l l l e a v e t h e o v e r a l l r e s u l t s enormously s i g n i f i c a n t (495 h i t s i n
4 ,000 t r i a l s , CR = 5.01, p<6x10'7, 2 - t a i l e d ) , b u t i t i s more u s e f u l and r e -
v e a l i n g t o examine t h e n a t u r e o f t h i s p o t e n t i a l , and t h e n c o n s i d e r t h e more
p e r t i n e n t q u e s t i o n s o f j u s t how much s c o r i n g can be g o t t e n from e f f i c i e n t ap-
p l i c a t i o n of s u c h p o t e n t i a l , and whether o r n o t t h e r e i s ev idence t h a t s u c h
a p o t e n t i a l was a c t u a l l y u t i l i z e d by t h e p e r c i p i e n t s .
Chi-Square Bias T e s t s Are Poor P r e d i c t o r Measures:
I f we t h i n k t h a t a p e r c i p i e n t migh t t a k e advantage o f b i a s e s he d i s c o v e r s
T a r t -10
th rough feedback i n a t a r g e t s o u r c e , i t i s p r o b a b l y common t o assume t h a t t h e
magni tude of t h e Chi-square measures ( o r D r . G a t l i n ' s D-measures) of d e p a r t u r e
from a n e q u i p r o b a b i l i t y and s e r i a l independence model i s a measure of how
p r e d i c t a b l e t h e sequence i s . T h i s i s i n c o r r e c t , a s t h e f o l l o w i n g examples w i l l
show.
Suppose a p e r c i p i e n t works i n a n exper iment i n v o l v i n g g u e s s i n g t h e num-
b e r s one t o t e n , i n a n exper iment f i x e d a t 200 t r i a l s . We s h a l l d e l i b e r a t e l y
use a b i a s e d t a r g e t s o u r c e , such t h a t o u t p u t s 1, 2, 3, 4 , and 5 a l l have a
p r o b a b i l i t y o f .15, w h i l e t h e o u t p u t s 6 , 7, 8 , 9, and 10 have a p r o b a b i l i t y
o f .05 , i n s t e a d o f a l l t a r g e t s b e i n g e q u i p r o b a b l e . The observed d i s t r i b u t i o n
of t a r g e t s i n t h e f i r s t h a l f of our exper iment (100 t r i a l s ) would l o o k l i k e
t h i s , d e l i b e r a t e l y g i v i n g i t a p e r f e c t r e f l e c t i o n o f t h e t a r g e t b i a s p a t t e r n
f o r s i m p l i c i t y o f i l l u s t r a t i o n
TARGETS 1 2 3 4 5 6 7 8 9
FREQUENCY 1 5 1 5 15 15 15 5 5 5 5 5
The s t a n d a r d Chi-square test f o r e q u a l f r equency o f o b s e r v e d - t a r g e t s
would t e l l us t h a t t h i s sample of 100 i s n o t from a random s o u r c e , a s we g e t
a Chi-square o f 25.00 w i t h 10 d e g r e e s o f freedom, p < . 0 1 , o n e - t a i l e d .
Suppose o u r p e r c i p i e n t t a k e s t h e f i r s t 100 t r i a l s t o c a t c h on t o t h i s
s i n g l e t b i a s p a t t e r n , s o t h a t w h i l e he has o n l y s c o r e d t h e 1 0 h i t s expec ted
under o u r assumed e q u i p r o b a b i l i t y model i n t h e f i r s t 100 t r i a l s , he w i l l u s e f
a mathemat ica l i n f e r e n c e s t r a t e g y , based on h i s new knowledge, f o r t h e remain-
i n g 100 t r i a l s o f t h e e x p e r i m e n t , H i s b e s t s t r a t e g y i s t o guess a 1, 2, 3,
4 , o r 5 on e v e r y t r i a l . I t does n o t m a t t e r whether he p i c k s one o f t h e h igh
f i v e and a lways g u e s s e s i t o r randomly a l t e r n a t e s among t h e h igh f i v e : we
would e x p e c t him t o s c o r e a b o u t 15 h i t s i n t h e second 1 0 0 . t r i a l s . I f we
a s s e s s t h e s i g n i f i c a n c e o f t h i s second-ha l f s c o r e under o u r a s sumpt ion of
e q u i p r o b a b i l i t y , w e compute a CR of 1.67, p < .05, o n e - t a i l e d . For t h e whole
exper iment o f 200 t r i a l s , we now have ( l W 1 5 ) =25 h i t s w i t h a n a s s o c i a t e d CR
T a r t -11
o f 1 .18, which, w h i l e n o t r e a c h i n g s t a t i s t i c a l s i g n i f i c a n c e , migh t s u g g e s t
t o a n exper imente r t h a t something was happening.
I f t h e exper iment was l o n g e r t h a n 200 t r i a l s t o t a l and t h e b i a s p a t t e r n
and mathemat ica l i n f e r e n c e s t r a t e g y were c o n s i s t e n t , t h e p e r c i p i e n t cou ld
o b v i o u s l y a t t a i n c o n v e n t i o n a l l e v e l s of s i g n i f i c a n c e a s he went f u r t h e r .
For 300 t r i a l s , f o r example, we would have (10+15+15)=40 h i t s , f o r a CR of
1 .92, p C . 0 5 , o n e - t a i l e d . We s h a l l s t a y w i t h a 200 t r i a l exper iment f o r
now, however, t o i l l u s t r a t e c e r t a i n p o i n t s .
Now c o n s i d e r a t a r g e t s o u r c e w i t h a q u i t e d i f f e r e n t s o r t o f b i a s , where
we obse rve t h e f o l l o w i n g d i s t r i b u t i o n of t a r g e t s i n t h e f i r s t 100 t r i a l s :
TARGETS 1 2 3 4 5 6 7 8 9
FREQUENCY 24 10 8 8 8 8 8 8 8
This g e n e r a t o r is h i g h l y b i a s e d toward p roduc ing o n e s , w i t h no o t h e r
l a r g e b i a s e s . The Chi-square t e s t f o r e q u i p r o b a b i l i t y o f s i n g l e t s f o r t h i s
d i s t r i b u t i o n g i v e s a v a l u e o f 22.40, p < . 0 2 , o n e - t a i l e d . I f we m i s t a k e n l y
assumed t h a t t h e magnitude o f t h e Chi-square v a l u e s r e f l e c t e d t h e degree o f
p r e d i c t a b i l i t y of t h i s and t h e p r e v i o u s t a r g e t s o u r c e s f o r a ma themat ica l in-
f e r e n c e s t r a t e g y , we would t h i n k t h i s second t a r g e t s o u r c e was e q u a l l y o r
s l i g h t l y l e s s p r e d i c t a b l e t h a n t h e s o u r c e i n t h e p r e v i o u s example. We shou ld
be q u i t e wrong.
t Again assume t h a t i t t a k e s o u r p e r c i p i e n t t h e f i r s t 100 t r i a l s t o c a t c h
on t o t h e b i a s , s o he s c o r e s o n l y 1 0 h i t s i n t h e f i r s t 100. Now he f o l l o w s
t h e o p t i m a l s t r a t e g y o f c a l l i n g a one f o r e v e r y one o f t h e remain ing 100
t r i a l s , and s c o r e s a b o u t 24 h i t s . For t h e second 100 t r i a l s a l o n e , t h i s
g i v e s a CR of 4 - 6 7 , P < o n e - t a i l e d . For t h e whole exper iment o f 200
t r i a l s , we have (10+24)=34 h i t s , w i t h a CR of 3.30, <.0005, o n e - t a i l e d .
For e q u a l Chi-square v a l u e s i n t e s t s o f b i a s , two sequences may d i f f e r
enormously i n u s e f u l n e s s f o r a m a t h e m a t i c a l i n f e r e n c e s t r a t e g y . F u r t h e r ,
T a r t -12
t h e r e w i l l be f a r , f a r more p o s s i b l e b i a s p a t t e r n s of l e s s u s e f u l n e s s f o r a
ma themat ica l i n f e r e n c e s t r a t e g y t h a n t h e r e w i l l be h i g h l y u s e f u l ones f o r a
g i v e n Chi-square v a l u e . There a r e many, many ways t o r e a r r a n g e t h e b i a s p a t -
t e r n i n our f i r s t example w i t h o u t g i v i n g i t t h e s i n g l e number peak b i a s p a t t e r n
o f o u r second example t h a t i s s o u s e f u l i n a ma themat ica l i n f e r e n c e s t r a t e g y .
Consider a t h i r d example where t h e f o l l o w i n g f r e q u e n c e s o f t a r g e t s a r e
obse rved i n t h e f i r s t 100 t r i a l s :
TARGETS 1 2 3 4 5 6 7 8 9 I FREQUENCY 1 8 10 9 9 9 9 9 9 9
The s t a n d a r d Chi-square test o f e q u i p r o b a b i l i t y t e l l s us t h a t t h i s is
n o t a b i a s e d sequence , f o r Chi-square i s o n l y e q u a l t o 7.20, which would occur - more than h a l f t h e t ime by chance a l o n e . Yet i f o u r t a r g e t s o u r c e i s r e a l l y
b i a s e d toward 18% ones i n t h i s way, and o u r p e r c i p i e n t d e c i d e s t o c a l l a l l
ones i n t h e second 100 t r i a l s of t h e exper iment , he cou ld make 18 h i t s t h e r e ,
f o r a second-hal f CR o f 2.67, p< .01 , o n e - t a i l e d , a n d a t o t a l o f (10+18)=28
h i t s f o r t h e whole exper iment , CR = 1.89, p< .05 , o n e - t a i l e d . It i s e s p e c i a l l y
i n t e r e s t i n g t o n o t e t h a t t h e e n t i r e sequence o f 200 t a r g e t s f o r t h i s p e r c i -
p i e n t , w i t h t h e 18% b i a s c o n t i n u i n g th rough t h e second 100 t r i a l s , s t i l l does
n o t show any s i g n i f i c a n t b i a s : Chi-square i s 14.40, p ) . 10 , o n e - t a i l e d .
We may conc lude t h e f o l l o w i n g f o r s t a n d a r d Chi-square t e s t s o f b i a s .
C For l o n g t o i n f i n i t e l e n g t h e x p e r i m e n t s ,
(1) Lack o f s i g n i f i c a n t Chi-square v a l u e s i n b i a s t e s t s p robab ly i n d i -
c a t e s l a c k o f s i g n i f i c a n t p r e d i c t a b i l i t y by a mathemat ica l i n f e r e n c e s t r a t e g y ;
and
(2 ) The p r e s e n c e o f s i g n i f i c a n t Chi-squares i n b i a s t e s t s i n d i c a t e s - some
d e g r e e o f p r e d i c t a b i l i t y by a mathemat ica l i n f e r e n c e s t r a t e g y , b u t t h e magni-
t u d e o f Chi-square does n o t i n d i c a t e t h e d e g r e e o f p r e d i c t a b i l i t y .
For s h o r t t o moderate l e n g t h e x p e r i m e n t s o f t h e t y p e f r e q u e n t l y c a r r i e d o u t ,
T a r t -13
however,
( 3 ) A s i g n i f i c a n t Chi-square i n d i c a t i o n o f , b i a s does - n o t n e c e s s a r i l y
i n d i c a t e t 'hat a s i g n i f i c a n t o v e r a l l s c o r e c a n be o b t a i n e d th rough a mathema-
t i c a l i n f e r e n c e s t r a t e g y ;
(4) The magni tu re o f t h e o b t a i n e d Chi-square is a v e r y poor measure
of t h e magni tude of r e s u l t s t h a t c a n be o b t a i n e d w i t h a ma themat ica l i n f e r e n c e
s t r a t e g y ; and
(5) A mathemat ica l i n f e r e n c e s t r a t e g y may produce s i g n i f i c a n t r e s u l t s
from a b i a s e d s o u r c e which does n o t appear t o be s i g n i f i c a n t l y b i a s e d by
Chi-square e v a l u a t i o n .
The shor tcomings of s t a n d a r d Chi-square measures o f b i a s i n r e a l i s t i c
l e n g t h exper iments i l l u s t r a t e why more d i r e c t measures o f p r e d i c t a b i l i t y by
ma t h e m a t i c a l i n f e r e n c e s t r a t e g i e s need t o be developed.
The f i n d i n g s o f s i g n i f i c a n t s i n g l e t and d o u b l e t b i a s i n a few of t h e
t a r g e t sequences used i n t h e f i r s t T r a i n i n g S tudy , t h e n , i n d i c a t e a p o t e n t i a l
f o r some k ind of ma themat ica l e s t i m a t i o n s t r a t e g y b e i n g employed, b u t do n o t
t e l l us e i t h e r ( a ) how much i t cou ld u s e f u l l y c o n t r i b u t e t o a r t i f a c t u a l l y
c r e a t i n g s i g n i f i c a n t r e s u l t s ; o r (b ) whe the r i t was a c t u a l l y employed; o r
( c ) how much such a s t r a t e g y c o u l d do compared w i t h t h e a c t u a l s c o r e s o b t a i n e d
by t h e p e r c i p i e n t s .
L e t us now c o n s i d e r t h e i m p o r t a n t d i f f e r e n c e between p r e d i c t i o n and pos t -
d i c t i o n .
Randomici ty , P r e d i c t i o n , and Pos t d i c t i o n :
P robab ly t h e most d i s h e a r t e n i n g a s p e c t o f s e v e r a l y e a r s o f exchanges
w i t h D r . G a t l i n i s h e r p e r s i s t e n t f a i l u r e t o comprehend t h e enormous d i f f e r -
e n c e between e d i c t i o n and p o s t d i c t i o n . I s h a l l q u o t e t h r e e pa ragraphs o f
a l e t t e r o f mine p u b l i s h e d e a r l i e r t h i s y e a r i n t h e J a n u a r y i s s u e of t h e
J o u r n a l o f t h e American S o c i e t y for P s y c h i c a l Research, p o i n t i n g o u t t h i s
problem.
T a r t -14
"Two meanings a r e g e n e r a l l y a s s o c i a t e d w i t h t h e concep t of r andomic i ty .
The f i r s t i s t h a t no p a t t e r n i n g s o r dependencies o f any s o r t c a n be found
i n a sequence o f random numer ica l d a t a . The second i s t h a t r andomic i ty
means a l a c k o f p r e d i c t a b i l i t y of a numerical sequence: t h a t i s , g iven
a sample o f t h e sequence, one cannot p r e d i c t subsequen t numbers i n t h e se-
quence w i t h g r e a t e r than chance s u c c e s s .
While t h e second meaning a s s o c i a t e d w i t h t h e concep t o f r andomic i ty
i s impor tan t f o r b o t h p s y c h o l o g i c a l and p a r a p s y c h o l o g i c a l r e s e a r c h , t h e
f i r s t i s f a l s e . Mathemat ica l ly , one c a n t ake any sequence o f numbers of any
f i n i t e l e n g t h , even i f t h e y have been g e n e r a t e d by a t r u l y random p r o c e s s ,
and f i n d a n a l g o r i t h m which would d e t e r m i n i s t i c a l l y g e n e r a t e t h a t e x a c t
sequence o f numbers. Th i s seems t o imply t h a t t h e sequence o f numbers was
n o t random, b u t r e s u l t e d d e t e r m i n i s t r i c a l l y from t h a t a l g o r i t h m , and t h u s
had a p a t t e r n t o i t t h a t cou ld be d e t e c t e d and made use o f . However, t h e
a l g o r i t h m s o determined w i l l not s u c c e s s f u l l y p r e d i c t f u r t h e r numbers ga thered
from t h e same random s o u r c e a t a l e v e l beyond chance expec tancy . To p u t i t
a n o t h e r way, we c a n always f i n d some kind o f p a t t e r n i n r e t r o s p e c t , a pro-
c e s s a k i n t o t h e p s y c h o l o g i c a l p r o c e s s o f r a t i o n a l i z a t i o n o r p r o j e c t i o n ,
b u t t h a t does n o t mean t h a t t h e sequence was a c t u a l l y genera ted i n t h a t
f a s h i o n o r t h a t i t i s p r e d i c t a b l e . "
t More t e c h n i c a l d i s c u s s i o n s o f t h e s e p o i n t s can be found i n C h a i t i n , 1975, and
Gardner, 19 . My le t ter con t inued :
"Thus t h e q u e s t i o n of whether D r . G a t l i n ' s post-hoc a n a l y s e s can f i n d
any kind o f p a t t e r n ( i n t h e s e n s e o f d e p a r t u r e s from p e q u a l i n g e x a c t l y one-
t e n t h ) i n my t a r g e t d a t a i s n o t r e a l l y t h e r e l e v a n t q u e s t i o n : such p a t t e r n s
can be found, t o v a r y i n g d e g r e e s , i n t h e d a t a o f any and e v e r y p s y c h o l o g i c a l
and p a r a p s y c h o l o g i c a l exper iment . The r e l e v a n t q u e s t i o n i s whether such
p a t t e r n i n g s , s e q u e n t i a l dependenc ies , o r b i a s e s e x i s t i n t h e t a r g e t d a t a
T a r t -15
t o a deg ree s t r o n g enough t o have a l lowed p e r c i p i e n t s i n t h e T ra in ing Study
t o f i g u r e o u t t h e s e b i a s e s a s they went a l o n g (not p o s t hoc) , and make use - of them t o boos t t h e i r s c o r e s t o a l e v e l h igh enough t o make unnecessary
t h e occur rence o f ESP a s a n exp lana t ion ." (Ta r t , 1978, p 8 2 ) .
D r . G a t l i n c la imed i n her JASPR l e t t e r ( G a t l i n , 1978a) t o which I was
responding t h a t he r "monotone" ( s i n g l e t ) s t r a t e g y s co red s i g n i f i c a n t l y w i th
e i g h t o f t h e t e n t a r g e t sequences i n my s tudy . A s I po in t ed o u t i n my r e p l y ,
t h i s s t r a t e g y a p p a r e n t l y c o n s i s t e d o f p o s t hoc coun t i ng of t h e f requency
of observed s i n g l e t s i n the e n t i r e sequence and t hen p r e t e n d i n g you had
c a l l e d t h a t h i g h e s t s i n g l e t f o r your e v e r y response! I n t h e r e a l world
p e r c i p i e n t s do no t have a l l t h i s d a t a u n t i l a f t e r t h e i r c a l l s a r e made, s o
D r . G a t l i n ' s p o s t d i c t i v e procedure i s q u i t e spu r ious . I gave an example
o f performing a G a t l i n monotone p o s t d i c t i v e s t r a t e g y on 25 random numbers
t aken from a random number t a b l e : I s co red s i x spu r ious h i t s , f o r a binomial
p r o b a b i l i t y of .03. D r . G a t l i n is s t r o n g on h igher o rde r b i a s e s : u s ing
a d o u b l e t monotone p o s t d i c t i v e s t r a t e g y o f t he same type , I s co red 10 h i t s
i n 25 t r i a l s , e t c . The h igher t h e l e v e l of t h i s p o s t d i c t i v e s t r a t e g y , the
h igher your s c o r e i s f o r a sequence w i t h any b i a s i n i t , o r even on a random
sequence.
Given t h e t o t a l f a l l a c i o u s n e s s o f any kind of p o s t d i c t i v e s t r a t e g y ,
I found i t hard t o b e l i e v e t h a t D r . G a t l i n would con t i nue t o use i t a f t e r
i t was po in ted o u t , b u t s h e has. I n he r l a t e s t p u b l i c a t i o n (Ga t l i n , 1978b)
i t i s now g iven t he impress ive sounding t i t l e of a 'Vaximal Markov-3 s t r a t e g y . "
To quote D r . ~ a t l i n ' s c u r r e n t paper , "It i s i n s t r u c t i v e t o c a l c u l a t e how
h igh t h e s u b j e c t s cou ld s c o r e i f t h e i r e s t i m a t e s were 100% accu ra t e . I f
w e count t he t r i p l e t f r e q u e n c i e s i n e ach i n d i v i d u a l t a r g e t sequence and use
t h e s e a s a b a s i s f o r a s imple guess ing s t r a t e g y , which we w i l l c a l l a maximal
Markov-3 s t r a t e g y symbolized a s MM3, where in t he s u b j e c t guesses the symbol
T a r t -16
mos t l i k e l y t o o c c u r , g i v e n t h e two p r e c e e d i n g symbols i n t h e t a r g e t , t h e
Z-scores range from abou t 1 7 t o 19 which i s s u b s t a n t i a l l y h i g h e r t h a n any
obse rved i n t h e exper iment ." ( G a t l i n , 1978c, p.13).
I n he r ~ e c e n t l e t t e r t o t h e J o u r n a l of t h e American S o c i e t y for P s y c h i c a l
Resea rch ( G a t l i n , 1978b), D r . G a t l i n i n s i s t s t h a t s h e does know t h e d i f f e r -
ence between p r e d i c t i o n and p o s t d i c t i o n , y e t s h e a g a i n g i v e s a p o s t d i c t i v e
s t r a t e g y a s a n example o f h e r knowledge! Now perhaps I ' m o ld - fash ioned and
c o n s e r v a t i v e , b u t t h e d i c t i o n a r y d e f i n i t i o n o f t h e v e r b " p r e d i c t " i s "TO
t e l l o r d e c l a r e beforehand . . .", b e i n g d e r i v e d from L a t i n r o o t s meaning
t o s p e a k a b o u t something b e f o r e i t happens. D r . G a t l i n j u s t g i v e s a n o t h e r
example o f what p e r c i p i e n t s might have done, g i v e n he r l a t e r knowledge, b u t
t h i s i s h a r d l y p r e d i c t i n g . Her examples remind me o f t h e newspaper columns
o f s t o c k marke t a n a l y s t s who always b r i l l i a n t l y e x p l a i n why t h e market be-
haved t h e way i t d i d l a s t week. These a n a l y s t s seldom make any money on
t h e marke t .
D r . G a t l i n f u r t h e r a r g u e s i n t h i s l e t t e r of r e s p o n s e t h a t I misunder-
s t o o k he r monotone g u e s s i n g s t r a t e g y , and d e c l a r e s t h a t ". . . i n e i g h t o u t
o f t h e 1 0 sequences t h e p r o b a b i l i t y (Dr. ~ a t l i n ' s i t a l i c s ) o f s c o r i n g s i g -
n i f i c a n t l y i s 10% t o 40% . . ." ( ~ a t l i n , 1978b, p. 296). What t h i s means
i n terms of a c t u a l d a t a i s t h a t if a p e r c i p i e n t had happened t o guess t h e
one c o r r e c t o u t o f t e n p o s s i b l e monotone s t r a t e g i e s r i g h t a t t h e s t a r t o f
h i s o r h e r r e s p o n s e s t h e y c o u l d have s c o r e d a t t h e (.05 l e v e l , 2 - t a i l e d .
As ide from t h e f a c t t h a t t h e r e a r e many more ways o f g u e s s i n g wrong w i t h
t h i s s t r a t e g y t h a n g u e s s i n g r i g h t w i t h i t , a s i m p l e i n s p e c t i o n o f t h e d a t a
would have r e v e a l e d t h a t no p e r c i p i e n t used s u c h a monotone s t r a t e g y !
F u r t h e r i n s p e c t i o n of t h e d a t a would have shown t h a t even i f t h e y had used
i t t o maximal advan tage , t h e i r t o t a l h i t s s c o r e s would have been enormously
less s i g n i f i c a n t t h a n t h e y a c t u a l l y were! If I had bought many s h a r e s of
T a r t -17
a c e r t a i n s t o c k l a s t week I would i n d e e d have been r i c h t h i s week, b u t . . . . As l o n g as we are on t h e u n p l e a s a n t s u b j e c t o f m e a n i n g l e s s s t a t i s t i c a l
p rocedures , , I s h o u l d comment o n t h e s o - c a l l e d " c o n t r o l " a n a l y s e s r e p o r t e d by
D r . G a t l i n t h a t presumably s u p p o r t h e r ma in t h e s i s . Fo r t h e s e a n a l y s e s s h e
matched a computer pseudo-random number g e n e r a t o r o u t p u t a g a i n s t t h e t a r g e t
s e q u e n c e s and t h e n a r t i f i c i a l l y c r e a t e d a number o f h i t s e q u a l t o t h o s e t h e
a c t u a l p e r c i p i e n t s made on e a c h t a r g e t s equence by l o o k i n g a t t h e t a r g e t l i s t
a t random i n t e r v a l s and s i m p l y c h a n g i n g t h e pseudo-random o u t p u t o f t h e com-
p u t e r g e n e r a t o r t o ma tch t h e t a r g e t and t h u s make a h i t . She r e p o r t s t h a t
t h e s e r e s p o n s e s e q u e n c e s showed none o f t h e s i g n i f i c a n t D-measures t h a t t h e
a c t u a l p e r c i p i e n t r e s p o n s e s e q u e n c e s d i d .
Her p r o c e d u r e amounts t o t a k i n g a v e r y s m a l l sample from a s l i g h t l y
b i a s e d sequence . A s m a l l s ample , o f c o u r s e , i s u n l i k e l y t o have a d e t e c t a b l e
b i a s i n i t s i m p l y b y t h e l a r g e r e d u c t i o n o f N. Then m i x i n g t h i s s m a l l sample
w i t h s e v e r a l t i m e s as many pseudo-random numbers d i l u t e s any b i a s even f u r t h e r .
It i s no wonder no b i a s e s were found. I n d e e d , t h e pseudo-random g e n e r a t o r
u s e d was p r o b a b l y o f t h e same t y p e t h a t D r . G a t l i n s t a n d a r d i z e d h e r D-measures
on i n t h e f i r s t p l a c e . I c a n n o t u n d e r s t a n d what meaning t h i s s o - c a l l e d con-
t r o l a n a l y s i s has .
Magni tude o f B i a s V e r s u s P a t t e r n o f B i a s :
I g n o r i n g f o r t h e moment t h e t r i p l e t and o t h e r f a l l a c i e s i n D r . G a t l i n ' s
a n a l y s e s , s u p p o s i n g we assumed t h a t h e r a n a l y s e s a t l eas t d e m o n s t r a t e d t h e
p o s s i b i l i t y t h a t a m a t h e m a t i c a l i n f e r e n c e s t r a t e g y m i g h t have been used b y
a t l e a s t some p e r c i p i e n t s . I f we l o o k a t t h e d a t a a v a i l a b l e t o D r . G a t l i n
t o see i f t h e y a c t u a l l y s u p p o r t t h i s p o s s i b i l i t y , we s h a l l s e e e v i d e n c e t h a t
t h e p e r c i p i e n t s d i d not u s e s u c h a s t r a t e g y .
To u s e a m a t h e m a t i c a l i n f e r e n c e s t r a t e g y , a p e r c i p i e n t s h o u l d p a t t e r n
h i s o r he r r e s p o n s e s t r a t e g i e s a s c l o s e t o h i s e s t i m a t e o f t a r g e t b i a s p a t t e r n s
T a r t -18
a s p o s s i b l e . We would t h e n e x p e c t t o s e e a correspondence between t h e most
f r e q u e n t b i a s p a t t e r n s i n t h e t a r g e t sequence, t h e t h i n g s t h a t would be most
u s e f u l f o r a n i n f e r e n c e s t r a t e g y , and t h e p e r c i p i e n t s ' r e sponse p a t t e r n s .
D r . G a t l i n c l a i m s t h a t such b i a s p a t t e r n matchings e x i s t , a s i n t h e opening
s t a t e m e n t of her d i s c u s s i o n s e c t i o n when s h e s t a t e s , "The matching p a t t e r n s
(my i t a l i c s ) demonstra ted i n t h e t a r g e t and guess sequences o f t h e s e two
independent sets o f ESP d a t a i n d i c a t e t h a t e x t r e m e l y s u b t l e b i a s a t h i g h
n - tup le l e v e l s i n f i n i t e sequences can be u t i l i z e d by the human mind."
( G a t l i n , 1978c, pp. 15-16). T h i s c l a i m , however, has no e m p i r i c a l s u p p o r t
a t a l l p r e s e n t e d f o r i t i n D r . G a t l i n ' s paper : she h a s examined t h e magnitude
o f b i a s e s , b u t p r e s e n t e d no d a t a a t a l l on whether t h e s p e c i f i c p a t t e r n s of
b i a s i n t a r g e t and p e r c i p i e n t d a t a a c t u a l l y match. I f a t a r g e t sequence is
h i g h l y b i a s e d toward t h r e e s f o l l o w i n g f i v e s , f o r example, and a p e r c i p i e n t
i s h i g h l y b i a s e d toward responding w i t h s i x e s a f t e r a t a r g e t has been f i v e ,
t h e s e h i g h magni tudes o f b i a s w i l l n o t be a t a l l u s e f u l f o r s c o r i n g , a s t h e
p a t t e r n s do n o t match.
One would have expec ted t h a t D r . G a t l i n would have i n s p e c t e d t h e a c t u a l
b i a s p a t t e r n s i n t h e t a r g e t and p e r c i p i e n t d a t a t o see i f they d i d match.
S i n c e she has e i t h e r n o t done s o o r chosen n o t t o p r e s e n t such d a t a , I c a r r i e d
o u t t h i s a n a l y s i s .
T I f a t a r g e t sequence had a h igh s i n g l e t l e v e l b i a s f o r e i g h t s , e . g . ,
we would e x p e c t t o s e e t h e p e r c i p i e n t showing most of h i s above-chance h i t s
on e i g h t s , r a t h e r t h a n o t h e r t a r g e t s . I f a d o u b l e t l e v e l mathemat ica l e s t ima-
t i o n s t r a t e g y was a l s o u s e f u l , t h e n we shou ld s e e many o f t h e above-chance
h i t s on t h e second term o f t h e d o u b l e t : i f n i n e s fo l lowed f i v e s v e r y f r e q u e n t l y ,
f o r example, we shou ld have many h i t s on n i n e s , a s w e l l a s on t h e e i g h t s ( i n
t h i s example) t h a t we have a l r e a d y d e f i n e d a s u s e f u l f o r a s i n g l e t e s t i m a t i o n
s t r a t e g y . We would n o t e x p e c t more than a chance number o f h i t s on t a r g e t s
T a r t -19
t h a t were n o t t h e h i g h ones i n a n e s t i m a t o r s t r a t e g y . How d o t h e d a t a a c t u a l l y
l o o k ?
Table ' 2 shows t h e t a r g e t numbers which had t h e h i g h s i n g l e t and h i g h
d o u b l e t o c c u r r e n c e s f o r t h e t e n p e r c i p i e n t s and t h e i r r e s p e c t i v e t a r g e t
s e q u e n c e s . The v a s t m a j o r i t y o f t h e s e d o n o t , o f c o u r s e , r e p r e s e n t s t a t i s -
t i c a l l y s i g n i f i c a n t b i a s e s . As c a n be s e e n , o n l y one p e r c i p i e n t had h i s h i g h e s t
s i n g l e t b i a s toward t h e same t a r g e t t h a t was h i g h i n t h e t a r g e t s e q u e n c e , and
no p e r c i p i e n t had h i s h i g h e s t d o u b l e t b i a s i d e n t i c a l t o t h e h i g h e s t d o u b l e t
b i a s i n h i s t a r g e t sequence . T h i s r e s u l t i s n o t s u p p o r t i v e o f Dr. G a t l i n ' s
c l a i m s . I n d e e d , i n s p e c t i o n o f t h e v a r i e t y o f h igh s i n g l e t s and d o u b l e t s
m i g h t l e a d us t o wonder how t h e same e l e c t r o n i c RNG c o u l d p roduce s u c h d i f -
f e r e n t p a t t e r n s f o r d i f f e r e n t p e r c i p i e n t s i f i t were as b i a s e d a s D r . G a t l i n
imp 1 ies .
I n s e r t Tab le 2 a b o u t h e r e
The d a t a i n T a b l e 2 do n o t c o m p l e t e l y d i s p r o v e t h a t t h e p e r c i p i e n t s migh t
have used a m a t h e m a t i c a l i n f e r e n c e s t r a t e g y , however, f o r t h e i r i n c o r r e c t h i g h
r e s p o n s e b i a s e s m i g h t have r e p r e s e n t e d p e r s o n a l i d i o s y n c r a c i e s , y e t t h e y
migh t have s t i l l p i c k e d up enough b i a s toward t h e t a r g e t s equence h i g h s t o
a r t i f a c t u a l l y p roduce t h e e x t r a - c h a n c e h i t s which made t h e i r s c o r e s s i g n i f i - s c a n t . A more d i r e c t test o f t h e h y p o t h e s i s i s shown i n T a b l e 3 . F o r s i m p l i -
c i t y , o n l y t h e f i v e p e r c i p i e n t s who s c o r e d s i g n i f i c a n t l y above chance a r e I
shown. The second column is t h e CR o f t h e i r obse rved h i t s , g i v e n t h e e q u i -
p r o b a b i l i t y model. To compute t h e CRs i n t h e t h i r d column, a l l t h e t r i a l s
(and, of c o u r s e , h i t s ) on t h e m o s t f r e q u e n t l y o c c u r r i n g s i n g l e t t a r g e t were
d e l e t e d --- f rom t h e t o t a l t r i a l s : i f t h e p e r c i p i e n t s were o n l y u s i n g a s i n g l e t
e s t i m a t o r s t r a t e g y , t h i s s h o u l d r e d u c e t h e i r s c o r e s t o chance . Obv ious ly i t
T a r t -20
d o e s no t . They were s c o r i n g q u i t e s i g n i f i c a n t l y on many o t h e r t a r g e t s t h a n
t h e one a s i n g l e t e s t i m a t o r s t r a t e g y would g i v e them a n advantage on. To
t e s t a n i n f e r e n c e h y p o t h e s i s e v e n f u r t h e r , a l l h i t s on t h e r e l e v a n t t a r g e t
p r e d i c t e d by t h e most f r e q u e n t t a r g e t d o u b l e t were a l s o d e l e t e d , y e t t h e C R s
i n t h e f o u r t h column show t h e p e r c i p i e n t s s t i l l c o n t i n u e t o s c o r e v e r y s i g -
n i f i c a n t l y .
I n s e r t Table 3 a b o u t h e r e
Thus s t r a i g h t f o r w a r d i n s p e c t i o n o f d a t a on t h e r e l e v a n t b i a s e s t i m a t e s
s t r o n g l y s u g g e s t s t h a t t h e p e r c i p i e n t s were not u s i n g a u s e f u l e s t i m a t o r
s t r a t e g y , c o n t r a r y t o D r . ~ a t l i n ' s c l a i m s .
G a t l i n ' s Opt imal E s t i m a t i o n Windows:
I f i n d D r . G a t l i n ' s s e c t i o n on o p t i m a l e s t i m a t i o n windows r a t h e r d i f f i -
c u l t t o f o l l o w . Presumably s h e is c l a i m i n g t h a t i f s i g n i f i c a n t and c o n s i s t e n t
b i a s p a t t e r n s e x i s t i n t h e t a r g e t sequence, a p e r c i p i e n t must d e t e c t what
t h e s e p a t t e r n s a r e e a r l y enough i n t h e exper iment t o be a b l e t o e f f e c t i v e l y
u s e them t o i n f l a t e h i s o r h e r s c o r i n g . A t l e a s t t h i s i s my r e a d i n g of what
would make s e n s e . The p rocedure D r . G a t l i n f o l l o w s t o presumably demons t ra te
t h a t t h i s was p o s s i b l e i s u n c l e a r t o m e , however. A t one p o i n t i t seems t o
i n v o l v e t h e c a l c u l a t i o n o f 300 c o r r e l a t i o n c o e f f i c i e n t s a s s h e g e t s h e r rnet r
v a l u e s f o r D;, D;, and D; f o r t e n i n c r e m e n t a l sample l e n g t h s over t e n p e r c i -
p i e n t s , and s u c h a l a r g e number o f c o r r e l a t i o n s seem bound t o y i e l d some s i g -
n i f i c a n t v a l u e s by chance a l o n e . The f i n a l outcome o f a l l t h i s i s even more
p u z z l i n g , however.
I f a p e r c i p i e n t a c t u a l l y f i g u r e s o u t a s i g n i f i c a n t b i a s p a t t e r n i n a
t a r g e t sequence and e x p l o i t s i t , what w e b a s i c a l l y e x p e c t t o s e e i s s c o r i n g
n e a r chance l e v e l f o r a w h i l e , b u t i n c r e a s i n g markedly and s t e a d i l y once t h e
b i a s p a t t e r n i s g rasped . Tha t i s , w e would e x p e c t a " l e a r n i n g " c u r v e , a
T a r t -21
s t e a d y i n c r e a s e i n s c o r i n g w i t h f u r t h e r t r i a l s . This e x p e c t a t i o n o f l e a r n i n g
is e s p e c i a l l y a p p l i c a b l e f o r h i g h e r o r d e r b i a s e s , where i t migh t t a k e some
t ime and a few p a r t i a l l y s u c c e s s f u l s t a r t s f o r t h e p e r c i p i e n t t o g e t i t r i g h t .
As you know, my pr imary i n t e r e s t i n t h i s d a t a was i n l o o k i n g f o r l e a r n i n g ,
and w h i l e I r e p o r t e d some e v i d e n c e f o r i t ( T a r t , 1975, 1976a), i t was h a r d l y
a s p e r v a s i v e a s we would e x p e c t from t h e , a p p l i c a t i o n o f s u c c e s s f u l mathemati-
c a l i n f e r e n c e s t r a t e g i e s by t h e p e r c i p i e n t s !
Consider t h e d a t a p r e s e n t e d by D r . G a t l i n i n he r Table 2 on o p t i m a l
es t i m a t i o n windows. For conven ience , I have p r e s e n t e d them g r a p h i c a l l y i n
F i g u r e Her procedure seems t o be a m a t t e r of c o r r e l a t i n g samples o f t h e
t o t a l t a r g e t d i s t r i b u t i o n f o r a p e r c i p i e n t w i t h t h e whole t a r g e t d i s t r i b u t i o n
o f t h a t p e r c i p i e n t . S i n c e t h e sample s i z e s s t a r t a s o n l y a s m a l l (5%) sample
b u t g e t i n c r e a s i n g l y b i g g e r a s t h e y inc rement , we would e x p e c t t h e s e c o r r e l a -
t i o n s t o g e t i n c r e a s i n g l y b i g g e r : a f t e r a l l , w e a r e c o r r e l a t i n g i n c r e a s i n g l y
more adequa te samples o f a d i s t r i b u t i o n w i t h i t s e l f . Yet D r . ~ a t l i n ' s own
a n a l y s e s show t h a t t h e s i n g l e t samples g e t b e t t e r f o r a w h i l e and t h e n g e t
worse: a s i d e from t h e mathemat ica l oddness o f t h i s , which makes m e s u s p e c t
some e r r o r , how i s t h i s supposed t o be h e l p f u l t o a p e r c i p i e n t ? The d o u b l e t
l e v e l c o r r e l a t i o n s s t a y r e l a t i v e l y c o n s t a n t a s soon as t h e sample s i z e in -
c r e a s e s above i t s i n i t i a l v e r y s m a l l s i z e , and t h i s i s p r o b a b l y main ly a r e -
f l e c t i o n o f t h e c o n s i s t e n c y of t h e e x p e r i m e n t e r e r r o r t h a t l e d t o t h e system- t
a t i c d e p l e t i o n o f XX d o u b l e t s , a n e r r o r I have a l r e a d y shown ( T a r t , 19772)
t o r e q u i r e o n l y a t r i v i a l c o r r e c t i o n o f t h e r e s u l t s . Perhaps someone e l s e
c a n f i g u r e o u t e x a c t l y what D r . G a t l i n d i d t o o b t a i n t h e s e r e s u l t s and what
s i g n i f i c a n c e , i f any, t h e y migh t have.
- - - - - - - - - - - - - - - - - - I n s e r t F i g u r e a b o u t h e r e
T a r t -22
Applying a Powerful I n f e r e n c e S t r a t e g y :
From t h e a n a l y s e s I have p r e s e n t e d , I have shown t h a t w h i l e some p o t e n t i a l
may e x i s t ' i n a s m a l l m i n o r i t y o f t h e t a r g e t sequences by which some s o r t o f
ma themat ica l i n f e r e n c e s t r a t e g y migh t have c o n t r i b u t e d t o a t l e a s t some p a r t
o f t h e obse rved r e s u l t s , D r . ~ a t l i n ' s c l a i m s a b o u t t h i s a r e i n v a l i d because
o f r a t h e r b a s i c p r o c e d u r a l and s t a t i s t i c a l f l aws . I n d e e d , i f s h e had in-
s p e c t e d t h e raw d a t a s h e based he r a n a l y s e s on, s h e would have s e e n t h a t t h e y
c o n t r a d i c t e d he r a s s e r t i o n s . As I ment ioned a t t h e beg inn ing o f t h i s paper ,
however, t h e g e n e r a l q u e s t i o n o f t h e e x t e n t t o which mathemat ica l i n f e r e n c e
s t r a t e g i e s c o u l d a f f e c t r e s u l t s i n feedback s t u d i e s i s a n i m p o r t a n t q u e s t i o n ,
s o l e t me b r i e f l y a d d r e s s i t d i r e c t l y by d e s c r i b i n g t h e r e s u l t s o f a power-
f u l i n f e r e n c e s t r a t e g y a p p l i e d t o t h e d a t a o f t h e f i r s t T r a i n i n g Study. Th i s
w i l l be f a m i l i a r t o some o f you a s I b r i e f l y touched on i t i n my P r e s i d e n t i a l
a d d r e s s ( T a r t , 1977b) l a s t y e a r .
The h y p o t h e s i s t h a t p e r c i p i e n t s c a n i n f l a t e t h e i r s c o r e s by a mathemati-
c a l i n f e r e n c e s t r a t e g y a s a r e s u l t o f f i g u r i n g o u t t a r g e t b i a s e s needs t o be
c a s t i n a s p e c i f i c and t e s t a b l e form t o be s c i e n t i f i c a l l y u s e f u l . F o r t u n a t e l y ,
ma themat ica l i n f e r e n c e l e n d s i t s e l f t o p r e c i s e d e f i n i t i o n . Eugene Dronek, a
c o l l e a g u e i n t h e Computer S c i e n c e s Department o f t h e U n i v e r s i t y o f C a l i f o r n i a
a t Berke ley , and I a r e now s u b m i t t i n g f o r p u b l i c a t i o n t h e r e s u l t s o f a v e r y
powerful ma themat ica l i n f e r e n c e s t r a t e g y t h a t we c a l l t h e P r o b a b l i s t i c Pre-
d i c t o r Program (PPP). I am p r i m a r i l y r e s p o n s i b l e f o r t h e b a s i c s t r a t e g y , and
Dronek i s p r i m a r i l y r e s p o n s i b l e f o r i t s p r a c t i c a l implementa t ion on t h e computer.
We s e t o u r s e l v e s t h e t a s k o f d e v i s i n g a compute r -ass i s t ed i n f e r e n t i a l
c a l l i n g s t r a t e g y t h a t would have enormously more power t h a n we c o u l d r e a s o n a b l y
a t t r i b u t e t o human p e r c i p i e n t s . We gave o u r program powers s u c h a s a n a b s o l u t e l y
p e r f e c t memory f o r a l l p r e v i o u s t a r g e t s t o d a t e , a l l p r e v i o u s t a r g e t d o u b l e t s ,
e t c . , up t o a l l p r e v i o u s t a r g e t s e x t u p l e t s , a s w e l l a s p e r f e c t l y a c c u r a t e and
T a r t -23
w e l l n i g h i n s t a n t a n e o u s ( i n terms o f human t i m e ) computing c a p a c i t y t o a s s e s s
p o s s i b l e b i a s e s .
To get a n overview o f what t h e PPP d o e s , assume t h a t t h e 1 0 1 s t t r i a l
i s coming up. To make i t s c a l l , our PPP i n f e r e n c e program l o o k s a t a l l
hundred p r e v i o u s t a r g e t s which have come up on p r e v i o u s t r i a l s . It has
a l r e a d y s o r t e d them i n t o a s i n g l e t f i l e , , a d o u b l e t f i l e , and s o on th rough
a s e x t u p l e t f i l e . I t l o o k s a t t h e s i n g l e t f i l e , a s k s what has been t h e most
f r e q u e n t s i n g l e t t o d a t e , and, g i v e n 100 t r i a l s , what i s t h e e x a c t b inomia l
p r o b a b i l i t y t h a t a s i n g l e t s h o u l d have appeared w i t h s u c h an obse rved £ r e -
quency compared t o t h e n u l l h y p o t h e s i s t h a t a l l s i n g l e t s have a n e q u a l pro-
b a b i l i t y o f one- ten th? Th i s e x a c t b inomia l p r o b a b i l i t y i s computed and s t o r e d .
The program t h e n a s k s i f t h e r e i s r e l e v a n t i n f o r m a t i o n i n i t s d o u b l e t f i l e :
t h a t i s , s a y t h e 1 0 0 t h t a r g e t was a 7 . Does t h e d o u b l e t f i l e have any i n f o r -
ma t ion on what 7s have been fol lowed by i n t h e p r e v i o u s 100 t r i a l s ? I f n o t ,
i t w i l l g u e s s on t h e b a s i s o f t h e most improbable (compared t o t h e n u l l hypo-
t h e s i s ) t a r g e t t o d a t e i n t h e s i n g l e t f i l e , b u t i f t h e d o u b l e t f i l e does have
r e l e v a n t i n f o r m a t i o n , i t w i l l a g a i n compute t h e e x a c t b inomia l p r o b a b i l i t y
o f t h a t many o r more d o u b l e t s having o c c u r r e d i n t h e 100 t r i a l s t o d a t e , com-
pa red t o t h e n u l l h y p o t h e s i s o f e q u a l p r o b a b i l i t y f o r a l l p o s s i b l e d o u b l e t s .
Th i s b inomia l p r o b a b i l i t y w i l l t h e n b e compared t o t h e b i n o m i a l p r o b a b i l i t y
o f t h e h i g h e s t s i n g l e t t o d a t e : i f t h e h i g h e s t d o u b l e t t o d a t e i s less pro- f
b a b l e , i.e., r e p r e s e n t s more of a d e p a r t u r e from t h e model o f s e q u e n t i a l i n -
dependence t h a n t h e h i g h e s t s i n g l e t t o d a t e r e p r e s e n t s a s a d e p a r t u r e from
t h e e q u i p r o b a b i l i t y model, t h e p r o g r a m w i l l u s e t h a t d o u b l e t i n f o r m a t i o n a s
t h e b a s i s of i t s g u e s s i n g s t r a t e g y . S i m i l a r l y , i f t h e r e i s a r e l e v a n t t r i p l e t ,
q u a d r u p l e t , q u i n t u p l e t , o r s e x t u p l e t , t h e most r a d i c a l d e p a r t u r e from t h e
model o f e q u a l p r o b a b i l i t y and s e q u e n t i a l independence w i l l be used a s a b a s i s
f o r t h e g u e s s i n g s t r a t e g y . On t h e 102nd t r i a l , a l l computa t ions w i l l be re-done
T a r t -24
b e c a u s e t h e r e i s now a d a t a b a s e o f 1 0 1 t r i a l s i n s t e a d o f 100 , e t c . , s o t h e
program c o n s t a n t l y u p d a t e s i t s e l f i n o r d e r t o g e t t h e maximum i n f o r m a t i o n
from a l l t'he m a t e r i a l t o d a t e . Because o f t h i s u p d a t i n g , i t i s q u i t e s e n s i -
t i v e t o l o c a l l y s h i f t i n g b i a s e s , as w e l l a s g e n e r a l b i a s e s .
Note t h a t o u r PPP program i s n o t based o n a n y a s s u m p t i o n s a b o u t r e p r e -
s e n t a t i v e samples o r t h e l i k e : i t d e a l s w i t h what h a s a c t u a l l y been o b s e r v e d
t o d a t e and a lways tries t o c a p i t a l i z e o n t h e s e o b s e r v e d f r e q u e n c i e s .
F i g u r e i s a compar i son o f wha t o u r i n f e r e n t i a l s t r a t e g y program, w i t h
a l l i t s a d v a n t a g e s , c a n d o on t h e t a r g e t s e q u e n c e s , compared t o t h e s c o r e s
o f t h e a c t u a l p e r c i p i e n t s o f t h e f i r s t T r a i n i n g S tudy . As you c a n see, t h e
PPP manages t o r e a c h s t a t i s t i c a l s i g n i f i c a n c e o n o n l y two o f t h e t e n t a r g e t
s e q u e n c e s , and i t i s g e n e r a l l y s c o r i n g w e l l be low t h e a c t u a l p e r c i p i e n t s '
s c o r e s . I n two c a s e s o f p e r c i p i e n t s who d i d n o t show i n d i v i d u a l l y s i g n i f i -
c a n t ESP s c o r e s , t h e i n f e r e n t i a l s t r a t e g y program d i d b e t t e r , a l t h o u g h i t
d i d n o t r e a c h s t a t i s t i c a l s i g n i f i c a n c e . I n d e e d , t h e PPP s c o r e d a t chance
- - - - - - - - - - - - - - - - - - ,
I n s e r t F i g u r e a b o u t h e r e
(CR = .15) on t h e t a r g e t s e q u e n c e o f t h e mos t s u c c e s s f u l p e r c i p i e n t (P3 ) ,
f o r t h e b i a s e s i n t h e t a r g e t d i s t r i b u t i o n w e r e n o t u s e f u l f o r p r e d i c t i o n ,
e v e n i f t h e y were s t a t i s t i c a l l y s i g n i f i c a n t . I n g e n e r a l , t h e PPP c o u l d g e t
o n l y a b o u t 30% as many h i t s as t h e a c t u a l p e r c i p i e n t s g o t o v e r t h e whole
s t u d y , and e v e n i f we a d j u s t e d t h e p e r c i p i e n t s ' s c o r e s downward a c c o r d i n g l y
o n a n h y p o t h e s i s t h a t t h e y a r o s e f rom b o t h ESP and a m a t h e m a t i c a l i n f e r e n c e
s t r a t e g y , t h e amount o f ESP i n t h e e x p e r i m e n t was s t i l l enormous.
D r . G a t l i n p u t s much e m p h a s i s o n h i g h e r b i a s e s i n s p e c u l a t i n g t h a t a
m a t h e m a t i c a l i n f e r e n c e s t r a t e g y was used by t h e p e r c i p i e n t s . I mus t n o t e
t h a t w e compared v a r i o u s l e v e l s o f s e n s i t i v i t y o f t h e PPP t o h i g h e r ' . l e v e l
T a r t -25
b i a s e s , r a n g i n g f rom a l l o w i n g i t t o o p e r a t e o n l y a t t h e s i n g l e t l e v e l a l l
t h e way up t o t h e s e x t u p l e t l e v e l . Adding l e v e l s above t h e s i n g l e t was o f
v e r y l i t t l e h e l p t o t h e PPP, as p r a c t i c a l l y a l l o f i t s s i g n i f i c a n t p e r f o r -
mances came from i t s u s e of s i n g l e t l e v e l b i a s e s : t h e r e s i m p l y were n o t
h i g h e r l e v e l b i a s e s , u p t o t h e s e x t u p l e t l e v e l , t h a t were o f p r a c t i c a l u s e
f o r a m a t h e m a t i c a l e s t i m a t o r s t r a t e g y . T h i s is p e r f e c t l y i n a c c o r d w i t h
t h e k i n d o f o p e r a t i o n we would e x p e c t from a n e l e c t r o n i c r o u l e t t e whee l t y p e
o f RNG, o f c o u r s e .
E v i d e n c e t h a t P e r c i p i e n t s Did Not Use a n I n f e r e n c e S t r a t e g y :
I c o u l d c o n c l u d e a t t h i s p o i n t t h a t less t h a n o n e - t h i r d maximum o f t h e
s c o r i n g c o u l d be a t t r i b u t e d t o t h e b e s t m a t h e m a t i c a l i n f e r e n c e s t r a t e g y we
have b e e n a b l e t o d e v i s e , and a l l o w t h a t c l a i m t o s t a n d u n t i l someone e m p i r i -
c a l l y d e m o n s t r a t e s t h a t some o t h e r p r e d i c t o r program a p p l i e d t o t h i s d a t a
d o e s b e t t e r . I stress e m p i r i c a l l y d e m o n s t r a t e s , f o r I t h i n k c o n c e r n w i t h
D r . G a t l i n ' s r e p e a t e d c l a i m o f f a n t a s t i c comput ing and p a t t e r n r e c o g n i z i n g
a b i l i t i e s o f t h e human mind, s u p p o r t e d by i n v a l i d a n a l y s e s , is a w a s t e o f
o u r t ime as i t s t a n d s . L e t t h e n e x t c l a i m a n t f o r a p r e d i c t o r s t r a t e g y
d e m o n s t r a t e t h a t i t p r e d i c t s .
I t h i n k I can go much f u r t h e r i n my c o n c l u s i o n , however, and c l a i m
t h a t t h e d a t a s t r o n g l y s u g g e s t t h a t e i t h e r no m a t h e m a t i c a l i n f e r e n c e s t r a t e g y
was used t o any s i g n i f i c a n t e x t e n t t h e p e r c i p i e n t s , o r , i f one was u s e d , --- f
i t was a c o n s i d e r a b l y less p o w e r f u l one t h a n Dronek and I have d e v i s e d , and
s o would l e a v e e v e n more t h a n 70% o f t h e h i t t i n g i n t h e f i r s t T r a i n i n g S t u d y
a t t r i b u t a b l e t o ESP. T h i s c o n c l u s i o n has a l r e a d y been s u g g e s t e d by t h e a n a l y s e s
o f t h e p a t t e r n s o f t a r g e t and r e s p o n s e b i a s I p r e s e n t e d e a r l i e r , which showed
t h a t t h e p e r c i p i e n t s ' s t r o n g r e s p o n s e b i a s e s were a l m o s t a lways d i f f e r e n t
from t h e s l i g h t b i a s e s i n t h e t a r g e t s e q u e n c e s . F u r t h e r s t u d i e s o f t h e in -
t e r n a l p a t t e r n i n g o f t h e d a t a g i v e even s t r o n g e r s u p p o r t t o t h i s c o n c l u s i o n .
T a r t -26
I n my P r e s i d e n t i a l Address l a s t yea r ( T a r t , 1977b) and e l sewhere ( T a r t ,
1977c) , I r e p o r t e d on t h e d i s c o v e r y o f a s t r o n g , n e g a t i v e r e l a t i o n s h i p be-
tween t h e magni tude of ESP h i t t i n g on t h e r e a l t i m e t a r g e t and t h e magni tude
o f h i t t i n g on t h e immediate f u t u r e ( + l ) t a r g e t , a f i n d i n g i n d e p e n d e n t l y s i g -
n i f i c a n t i n two s e p a r a t e s t u d i e s . I proposed a t h e o r y o f t r a n s t e m p o r a l i n h i -
b i t i o n , a n i n f o r m a t i o n p r o c e s s i n g mechanFsm f o r ESP, t o accoun t f o r t h i s re-
l a t i o n s h i p , and I a l s o p o i n t e d o u t t h a t t h e f i n d i n g was q u i t e r o b u s t : i f
a l l t h e t a r g e t sequences showing s i g n i f i c a n t s i n g l e t b i a s ( t h r e e , i n two
s t u d i e s ) were d e l e t e d from t h e computa t ions , t h e r e l a t i o n s h i p was s t i l l
s t r o n g l y and s i g n i f i c a n t l y p r e s e n t . F i g u r e shows a t y p i c a l example of
t h e t empora l d i s p l a c e m e n t p a t t e r n o f h i t t i n g and m i s s i n g w i t h a t a l e n t e d
p e r c i p i e n t , w i t h s t r o n g r e a l time h i t t i n g , s t r o n g m i s s i n g on t h e +1 f u t u r e
t a r g e t and on t h e -1 and -2 p a s t t a r g e t s (due a t l e a s t p a r t l y t o r e s p o n s e
b i a s e s ) , and g e n e r a l p o s i t i v e and n e g a t i v e , l a r g e l y chance, f l u c t u a t i o n s f o r
o t h e r t i m e d i s p l a c e m e n t r e g i s t e r s .
As a c o n t r o l on a p o s s i b l e mathemat ica l e s t i m a t i o n s t r a t e g y a r t i f a c -
t u a l l y c r e a t i n g t h e n e g a t i v e r e l a t i o n s h i p between r e a l t ime and +1 h i t t i n g ,
I I r a n t h e same s o r t s o f a n a l y s e s f o r temporal d i s p l a c e m e n t s on t h e r e s p o n s e s
o f t h e PPP t o t h e t a r g e t sequences . F i g u r e is a temporal d i s p l a c e m e n t
a n a l y s i s on t h e same t a r g e t sequence a s t h a t used i n F i g u r e , and i t i s
t y p i c a l o f t h e PPP r e s u l t s . I c o u l d t a b u l a t e v a r i o u s pa ramete r s o f t h e s e
a n a l y s e s s t a t i s t i c a l l y and show enormous d i f f e r e n c e s , b u t t h e two f i g u r e s
convey t h e e s s e n t i a l p o i n t : a ma themat ica l e s t i m a t i o n s t r a t e g y l i k e t h e PPP
g i v e s i n t e r n a l p a t t e r n i n g t o t h e d a t a t h a t i s o b v i o u s l y n o t h i n g l i k e t h a t
shown by a c t u a l p e r c i p i e n t s . The d i f f e r e n c e s a r e d i s c u s s e d i n more d e t a i l
T a r t -27
e l sewhere ( T a r t , 1977b).
- - - - - - - - - - - - - - - - - - -- 5- I n s e r t F igure about here
SL toe R I
Any proponent of a mathemat ical e s t i m a t i o n theory , then , has a cha l l enge
here : what k ind o f p r e d i c t i v e s t r a t e g y can t hey d e v i s e t h a t can be empi r i -
c a l l y demonstra ted t o bo th produce t h e enormous number o f h i t s t h e pe r c i -
p i e n t s showed and t h e same i n t e r n a l p a t t e r n i n g s ?
Conclusions :
Time does no t a l l ow me t o adequa t e ly review the v a r i o u s p o i n t s touched
on i n t h i s paper , nor do I wish t o dwel l f u r t h e r on t h e inadequac ies o f D r .
G a t l i n ' s arguments and ana ly se s . I s h a l l c l o s e by j u s t ment ioning t h e main
p o s i t i v e c o n t r i b u t i o n s and conc lu s ions t h a t have come o u t o f t h i s d i s c u s s i o n .
F i r s t , t h e p o t e n t i a l importance o f i n f l a t i n g s c o r e s through some k ind
of mathemat ica l i n f e r e n c e s t r a t e g y i n ESP s t u d i e s employing immediate feed-
back of r e s u l t s has heen underscored.
Second, t h i s d i s c u s s i o n has emphasized t h a t t h e p r e d i c t a b i l i t y o f a
t a r g e t sequence i s our pr imary concern i n t h i s m a t t e r , no t d e p a r t u r e s from
randomness per se.
Thi rd , t h e inadequacy of t h e s t a n d a r d measure f o r b i a s , t h e Chi-square
test (and D r . G a t l i n ' s D-measures) f o r t e l l i n g us how p r e d i c t a b l e a t a r g e t ?
sequence i s by mathemat ica l i n f e r e n c e s t r a t e g i e s has been demonstra ted empiri-
c a l l y : I am s u r e some o f you who a r e more ma thema t i ca l l y i n c l i n e d than I
could demonstra te i t i n e l e g a n t mathemat ica l form.
Fourth , t h e need t o d e a l d i r e c t l y and e m p i r i c a l l y w i t h t he q u e s t i o n
o f p r e d i c t a b i l i t y of t a r g e t sequences , r a t h e r t han i n f e r e n t i a l l y , has been
underscored. The c o n t r i b u t i o n of Dronek and myself t h a t has been submi t ted
f o r p u b l i c a t i o n i s a powerful s t a r t i n d i r e c t l y d e a l i n g w i t h t h i s ques t i on .
T a r t - 28
F i f t h , a s t o t h e s p e c i f i c q u e s t i o n o f whe the r t h e p e r c i p i e n t s i n my
f i r s t T r a i n i n g S t u d y a c t u a l l y i n f l a t e d t h e i r o u t s t a n d i n g h i t t i n g s c o r e s b y
a m a t h e m a t i c a l i n f e r e n c e s t r a t e g y , i n a d d i t i o n t o o r i n s t e a d o f u s i n g ESP,
s e v e r a l a n a l y s e s s k e t c h e d h e r e have d e m o n s t r a t e d t h a t t h e y c o u l d n o t have
r done anywhere n e a r l y a s w e l l as t h e y d i d w i t h a known e s t i m a t o r s t r a t e g y ; - t w k b ~ ,
t h e e v i d e n c e s t r o n g l y s u g g e s t s t h e y d i d use a known e s t i m a t o r s t r a t e g y
t o any s i g n i f i c a n t e x t e n t .
The r e s u l t s o f t h i s f i r s t T r a i n i n g S t u d y have now been t h o r o u g h l y ques-
t i o n e d on a wide v a r i e t y o f grounds i n t h e l i t e r a t u r e ( G a t l i n , 1978a; 1978b;
Gardner , 1977; O ' B r i e n , 1976; S t a n f o r d , 1977) , and I b e l i e v e t h e r e s u l t s
i n d i c a t i n g t h e p r e s e n c e o f v e r y h i g h l e v e l s o f ESP i n t h e d a t a have w i t h -
s t o o d t h i s q u e s t i o n i n g e x t r e m e l y w e l l (see T a r t , 1976b; 1977d; 1977f ; 1978a)
making them some o f t h e b e s t d a t a i n con tempora ry p a r a p s y c h o l o g i c a l e x p e r i -
m e n t a t i o n . They s u g g e s t , among o t h e r t h i n g s , t h a t ser ia l s e l e c t i o n p r o c e d u r e s
c a n f i n d v e r y h i g h s c o r i n g p e r c i p i e n t s , t h a t immediate f eedback c a n a t l e a s t
s u s t a i n i f n o t i n c r e a s e ESP f u n c t i o n i n g , and t h a t we now have a g l i m p s e o f
a b a s i c i n f o r m a t i o n p r o c e s s i n g s t r a t e g y f o r ESP, t r a n s t e m p o r a l i n h i b i t i o n ,
which i n t u r n l e a d s t o a more s e n s i t i v e test f o r t h e p r e s e n c e o f ESP ( T a r t ,
1977b) . I would s u g g e s t t h a t t h e r e i s more p r o f i t i n t r y i n g t o r e p l i c a t e 4 5 h r q c fw*[
and expand t h e s e f i n d i n g s i n new e x p e r i m e n t a t i o n t h a n i n &
Tart -29
References
Cha i t in , G., Randomness and mathemat ica l proof. S c i e n t i f i c American, 1975,
'232, 47-52.
Davis, J., & Akers, C., Randomization and t e s t s f o r randomness.
J o u r n a l o f Parapsycholoqy, 1974, 38, %id 393-407.
Ga t l i n , L., Comments on t h e c r i t i c a l exchange betweeen Drs. S tan fo rd and
Tart . Jou rna l o f t h e A m e r i c a n Soc ie t y f o r P s y c h i c a l Research, 1978,
72, 77-81. (a )
G a t l i n , L., Dr. G a t l i n t s r e p l y t o Dr. Tart . J o u r n a l o f t h e American So-
c i e t y f o r P s y c h i c a l Research, 1978, 72, 294-296. ( b )
G a t l i n , L., A new measure o f b i a s i n f i n i t e sequences w i t h a p p l i c a t i o n s t o
ESP data. Paper, Parapsycho log ica l Assoc ia t ion , S t . Lou is , 1978. ( c )
Gardner, fl.,
Gardner, Me, ESP a t random. New York Review o f Books, 1977, August 14.
OtBr ien, D., Review of T a r t ' s " A p p l i c a t i o n o f .i?S Learn ing Theory t o ESP
Performance", Journa l o f Parapsycholoqy, 1976, 40, 76-81.
Stanford, R e , The a p p l i c a t i o n o f l e a r n i n g t heo ry t o ESP performance: a rev iew
o f Dr. C. T. T a r t ' s monograph. Jou rna l o f t h e American Soc ie ty f o r Psy-
c h i c a l Researbh, 1977, 71, 55-80.
! Tar t , C., Card guessing t e s t s : l e a r n i n g paradigm o r e x t i n c t i o n paradigm?
Jou rna l o f t he American Soc ie t y f o r Psych i ca l Research, 1966, 60, 46-55.
Tar t , C., The A p p l i c a t i o n o f Learn inq Theory t o ESP Performance. New York:
Parapsychology Foundation, 1975.
Tart -30
T a r t , C., L e a r n i n q t o Use E x t r a s e n s o r y - I J e r c e p t i o n . Chicago: U n i v e r s i t y o f
Ch icago P r e s s , 1976. ( a )
T a r t , C., Rqply t o O tBr i en . J o u r n a l o f P a r a p s y c h o l o q y , 1976, 40 , 240-246. ( b )
T a r t , C., Improv ing s r e a l - t i m e ESP by s u p p r e s s i n g t h e f u t u r e : t r a n s - t e m p o r a l
i n h i b i t i o n . P a p e r , I n s i t t u t e o f E l e c t r i c a l and E l e c t r o n i c E n g i n e e r s ,
New York, 1977. ( a )
T a r t , C., Space , time, and mind. P r e s i d e n t i a l Adres s , P a r a p s y c h o c l o g i c a l
A s s o c i a t i o n , Washington , D.C., 1977. ( b )
T a r t , C., P s i : S c i e n t i f i c S t u d i e s o f t h e P s y c h i c Realm. New York: Du t ton ,
1977. ( c )
T a r t , C., Toward h u m a n i s t i c e x p e r i m e n t a t i o n i n p a r a p s y c h o l o g y : A r e p l y t o
t o D r . S t a n f o r d . J o u r n a l o f t h e American S o c i e t y f o r P s y c h i c a l Resea rch ,
1977, 71 , 81-102. ( d )
T a r t , C., Toward c o n s c i o u s c o n t r o l o f p s i t h r o u g h immed ia t e f e e d b a c k t r a i n -
i n g : some c o n s i d e r a t i o n s o f i n t e r n a l p r o c e s s e s . J o u r n a l of t h e American
S o c i e t y f o r P s y c h i c a l R e s e a r c h , 1977, 71 , 375-408. ( e )
T a r t , C., P s i and s c i e n c e . New York Review o f Books, 1977, O c t o b e r 13. ( f )
T a r t , C., D r . T a r t ' s r e p l y t o D r . G a t l i n . J o u r n a l o f t h e American S o c i e t y
f o r P s y c h i c a l R e s e a r c h , 1978, 72 , 81-07.
Table 1
Bias Measures o f Target Sequences
i n t he F i r s t Training Study
S ing l e t Gat l in , Doublet Gat l in
Perc ip ien t
P5
P3
P4
* i nd i ca t e s P c .05 , l - t a i l ed .
T a b l e 2
High F r e q u e n c i e s o f T a r g e t G e n e r a t o r
V e r s u s P e r c i p i e n t s
Percipient Target Response Target Response
P5 7* 7* 895 7,5*
P3 5* 2* 995* g92*
P4 9 7* 599 7,9*
P2 9 5 5,9* 299
* indicates that averall distribution departed from the equiprobable or serial independence
model with P < .05, 1 -tailed.
+ ties for highest rank were broken randomly
Table 3 . .
Extreme Test of Mathematical Predictive
Strategy: Deleting Hits from Best
Predictors as - Postdicted, Singlet
and Doublet Levels
Deleting Deleting Hits on Hits on
Original Highest Highest Hits Singlet: Singlet&Doublet
Percipient C R* CR CR
* These CRs may differ slightly from published data due to
Gatlin1s practice of substituting random responses for
Pass data.
EXPECTATION
P< .01, ONE-TAILED
1 2 3 4 5 6 7 8 9 1 0 TARGETS . c E - T i -
EXPECTATION
x:,,, = 22.40 I
P < .O2, ON E-TAI LED
1 2 3 4 ' 5 6 7 8 9 10 TARGETS
------ ---------- EXPECT.ATION 1 0 9 9 9 9 9 9 9 9
x : ~ ~ , = 7.20
NS
2 3 4 5 6 7 8 9 10 TARGETS
G A T L I N ' S ' O P T I M A L E S T I M A T I O N W I N D O W S
ESTIMATION WINDOW LENGTH?
I 1 I I
- -
- - DOUBLETS
-
- -
- - \
- -
-
-
- -
I I I
B i a s Measures of Target Sequences
i n t h e F i r s t Tra in ing s tudy '
S i n g l e t Ga t l in Doublet Ga t l in
P e r c i p i e n t ,x2 X' ~ ; ( f )
* i n d i c a t e s P c.05, 1- ta i led .
P e r c i p i e n t
r 5
P3
F4
P2
High Biases of Target Generator
versus P e r c i p i e n t s
+ S i n g l e t s
Target Response
7* 7*
Response
7,5*
9,2*
7,9*
299
5,3*
899
2 ,8*
498
4,6*
5,6*
* i n d i c a t e s t h a t o v e r a l l d i s t r i b u t i o n departed
from t h e equiprobable o r s e r i a l independence
model with P < ,05, 1 - t a i l e d ,
+ t i e s f o r h ighes t rank were broken randomly
Extreme Test of Mathematical Predictive
Strategy: Deleting Hits from Best
Predictors as - Postdicted, Singlet
and Doublet Levels
Deleting Deleting Hits on Hits on
Original Highest Highest Hits Singlet : Singlet&Doublet
Percipient C R* CR CR
* These CRs may differ slightly from published data due to Gatlint s practice of substituting random responses for
Pass data.
