CommunicationDefinedasComplementaryInformative Processes

�

RobertM. LoseeSchoolof InformationandLibrary Sci.

U. of NorthCarolina-ChapelHillChapelHill, NC 27599-3360

U.S.A.losee@ils.unc.edu

Journalof Information,CommunicationandLibrary Science5 (3) 1999,pp1–15.

September28,1999

Abstract

Givena setof requirementsfor a definitionof communication,we candefinea communicationasinformationthatentersa processandeventuallyleavesits inverseprocess.For example,informationis transmittedby speak-ing andreceivedafterprocessingby its inverse,hearing.Thisdefinitioncanbeusedto preciselydescribeandexplain communicationphenomenain aninclusive andexactmanner. Thenatureof processesandtheir developmentis considered.Communicationprocessesmay supportotherprocesses,in-cluding non-communicative, evolutionarily adaptive processessupportingsurvival andreproduction.Communicationis expectedto develop in self-organizingsystems,givencertainassumptions.Receiving processesmaybeunderstoodasinformationfiltersandtheirperformancedescribed,predicted,andunderstood.Theseprecisedefinitionsof communicationandinforma-tion canserve asthebasisfor a scienceof librarianship.

�Theauthorwishesto thankDianeSonnenwaldandthemembersof herCommunicationSem-

inar for valuablecommentson theideaspresentedhere,aswell asElfredaChatmanfor conversa-tionsaboutdefininginformation.

1

1 Intr oduction

Communicationoccursonly whentherearetwo associatedinformationproducingprocessesandthe outputfrom oneprocessis the functionalinverseof the otherprocess’s output. We cansaythatcommunicationoccurredandthat informationwastransferredbetweentheinputto thefirst processandtheoutputof thesecond.Notethatcommunicationandinformationarenotsynonymousterms.Theformalincorporationof informationandprocessinto adefinitionof communicationpro-videsa modelof communicationthatcapturesmuchof thecommonsensemean-ing of communicationwhile allowing usto bothaccuratelypredictandpreciselyexplainagreatdealaboutcommunicationsystems.Dervin [Der93],in discussingthe disciplineof communication,notesthat “It is process,however—the verbsof communicating—wherewe have somethingto offer that is, if not ultimatelyunique,at leastfor now aheadof theothers.”

How do we study“what happensin the elusive momentsof humancommu-nicatings” [Der93], andwhat are “communicatings”?Whencommunicationisdefinedin termsof informativeprocesses,onecanstudyboththeinformationthatis conveyedandthe processesthat carry it. Definitionsof communicationofteninvolve termssuchasknowledge, belief, meaning, or intention. Moving beyondterminologyaboutwhich many epistemologistsandcognitive psychologistsdis-agreeto a definitionthat is oftenconsistentwith theseabstractideasbut is basedon morepreciseconceptsenablesusto useandbuild on thedefinition,aswell asmeasuretheoutputof communicatingphenomena,bothhumanandnon-human.Humancommunication,asubsetof all communication,playsaspecialrole in thestudyof communication,but humancommunicationis not equivalentto commu-nication,andthe formerneedsto studiedasa subsetof the latter. A precisebutgeneraldefinitionof communicationbasedon informationrequiresa preciseanddisciplineindependentdefinitionof information[Los97]. This will be providedbelow.

Thestudyof communicationphenomenamaybedividedinto theexperimentalandthetheoretical.Experimentalstudiesof communicationphenomenaexaminehow informative processeswork andpreciselywhat informationis provided bythem. Statementsare particularto a specificsituationor domainand general-izability often becomesproblematic. Theoreticalstudiesof communication,ontheotherhand,addresson a moreabstractlevel thenatureof processesandtheiroutputs,aswell asthe inter-relationshipsbetweenprocessesandtheir inputsandoutputs. Mentioning inputs and outputsbrings to mind introductorycomputerclasseswherethis terminologyis often introduced.Theoreticalcommunicationstudiesoftenuseterminologysimilar to thatusedin computerscience,with bothdisciplinesborrowing terminologyfrom the interdisciplinarytheorydevelopingin anareaintersectingmathematics,computerscience,communications,anden-gineering.Processand informationarebasedon conceptsin this intersectionof

2

disciplines.Why shouldonebeinterestedin aprecisedefinitionof communication?Hav-

ing a usefuldefinitionor modelof a phenomenonallows oneto describeobser-vationsin a consistentway. In addition,sucha modelmayallow usto accuratelypredict the characteristicsof previously unseencommunicationsystems.Mostimportantly, asuccessfuldefinitionandmodelhelpsusto explainhow thesystemoperates,leadingoneto anunderstandingof thenatureof thesystem.

A satisfactorydefinitionof communicationshouldpreservesomeof theideasin thecommonunderstandingof theterm. Sucha definitionshouldbeconsistentandincludeonly thoseitemsthatneedto beincluded,omitingmentionof aspectsthatneednot beincluded.At thesametime,a definitionshouldallowsusto bothbetterunderstandwhat communicationis, definingclearly what phenomenaareto be includedand which are to be excludedfrom the domainof communica-tivephenomena,aswell asto measureandevaluatecommunicationsystemsbothquantitatively andqualitatively. Qualitative studiesemphasizeunderstandingina way that demandsprecision,andwe believe that this definition brings to thedisciplineof communicationa precisionthat is necessaryandsufficient for bothqualitative andquantitative studies.The increasedunderstandinggainedfrom aprecisedefinitioncomesfrom statementsof what is andwhat isn’t communica-tion, aswell ashow a communicative systemfunctions. Being ableto describehow communicationtakesplaceallows one to manipulatesystemcomponents,improving thequantityor qualityof communicationtakingplace.A precisestate-mentof thefunctionof a systemalsoleadsto improvedevaluativemethods,withfeedbackfrom theseleadingto improvedsystems,andsoforth. A clearstatementof thedomainof studyallowsoneto focusonaphenomenonthatis definitelypartof communication,andthusworthy of studyin depth,or onemayaddressotherphenomenawith theunderstandingthatthey areclearlylesscentralto thefield.

2 Information

Informationhashistorically beenunderstoodandmeasuredin a wide varietyofways.Usuallythesemodelsaredisciplinespecificor limited in scope.For exam-ple, physicistsspeakof informationin thermodynamicterms,while epistemolo-gistsdescribeinformationassomethingthat occurswithin the context of higherlevel humancognitive processes.We briefly presentherea numberof differentideasaboutinformation,concludingwith a discipline independentdefinition ofinformationthatcanbeusedto provideabasisfor ageneraldefinitionof commu-nication.

Themostwidely understoodnotionof informationfor Englishspeakingchil-drenmaybeseenin CookieMonster’sdefinitionof informationas“newsor factsaboutsomething.” Otherdefinitionstendto be moreexplicitly human-centered,suchas“information is knowledge.” Similarly, Dretske [Dre81] views informa-

3

tion assomethingthat bringsus to certainty, a definition that annoys thosewhouseprobabilitytheoryandassumethatonecannever beabsolutelycertainaboutanything. More formal thanthis is Bar Hillel’ s modelof informationaswhat isexcludedby astatement[BHC53, BH55]. BarHillel’ sdefinitionprovidesabridgebetweenformal andrigorousdefinitionsof informationandtheideaof meaning.Baseduponearlierthermodynamicmodelsof entropy, Brillouin [Bri56] suggeststhat“information is a functionof theratio of thenumberof possibleanswersbe-fore and after....” and that anything that reducesthe size of the set of answersprovidesinformation.Shannon’s modelof informationandcommunication,withwhich we assumethe readeris familiar, measuresinformationasinverselypro-portionalto theprobabilityof asignal[Sha93b,Rit86, Rit91,Los90]. Mostof themeasuresof informationproducenumberssimilar to thoseproducedby thefamil-iar Shannonmodel,with theamountof informationbeinginverselyproportionalto theprobabilityof anevent.

Severalmeasuresof complexity andinformationexplicitly addressprocesses.Forexample,severalscholars,includingKolmogorov [Kol65], Solomonoff [Sol64a,Sol64b], andChaitin [Cha77],have separatelydevelopedmeasuresof the com-plexity or informationinherentin aprocessasthesmallestalgorithmthatproducestheoutputproducedby theprocessin question.TheMinimal Description(MD)of the process,with all redundancy andextraneousmaterialremoved, capturesthe essentialnatureof the process.Slightly different thanthis is the MinimumDescriptionLength(MDL), the lengthof the smallestdescriptionfor a process,proposedby Wallace[WF87,WB68] andRissannen[Ris89].

One measureof information, seeminglymore popular in the U.K. than inNorth America, measuresinformationas inverselyrelatedto the varianceof avariable[Fis25,Mac69]. As we learnmoreabouta variableor anevent(e.g. theaverageheightof communicationscholars)thevarianceof ourestimatedecreasesandtheinformationincreases.

Thereis a largegapbetweenwhat is providedby a measureandwhat is pro-videdby adefinition,andmany of themeasuresdevelopedin the“hardest”of thesciencesdonotexplicitly provideanassociateddefinition.Informationscientists,for example,may proposea model of an informationrelatedprocessand thenmeasurea characteristicof themodelwithout necessarilyproviding a cleardefi-nition of their terms.Definitionshelpclarify theessenceof aphenomenon,whilea measuremaycapturewhat is importantin somerespectaboutthephenomena.A measurewithoutadefinitionis still useful,howeveradefinitionwould improvethe understandingof the context for mostmeasures.We proposea definitionofinformation,followedby a definitionof communication,but onemayalsothinkof eithermoregenerallyasamodelallowing usto measure,describe,andpredictcharacteristicsandrelationships.

Informationand communicationmay be definedin relationshipto the pro-cessesthatmove theinformationfrom thebeginning,or entryof theinformation

4

into thechannel,to its exit from thechannel.A processis anaction-mediumthatmovesaninputpresentedto theprocessto theoutput,possiblyassigningnew val-uesto oneor moreoutputvariables.Weusethetermprocessbecauseof Church’sthesis,widely believed to be correct[KMA82], which suggeststhat processes,functions,andalgorithmsall describethesamephenomenon;all thesemodelsofcomputationfor mostpurposesareequivalentandonemaydescribeusingoneoftheseandat thesametimesatisfactorilyalsodescribeaseffectively asif onewereusingoneof theothermodels.

Informationmaybedefined[Los97] as

the valuesfor all variablesin the outputof any process.This infor-mationis abouteitherthe process,or the inputs to the process(thecontext of theprocess),or both.

A processis a setof relatedcomponentsthat produceor changesomething.Whenstudyinganinformationsystem,theremaybeonelargeprocessor severalsmallerprocesses.Thescopeof theprocessis not importantto our definitionofinformation,or later to our definition of communication.However, the choiceof how to view a processwill affect how onemay studyor understandthe pro-cess.For example,thereadermayview theentireprocessthat takestheauthor’sthoughtsandresultsin thethoughtsbeingunderstoodby thereaderasacommuni-cationprocess,or onemayview numeroussmallstepsin thewriting, publishing,andreadingprocessesaslinkedprocesses.Whetherviewedasonelargeprocessor several smallerprocesses,the informationpresentedat the endof the entirelargeprocessor at theendof thesetof smallerprocessesis thesame.

Theamountof informationproducedby a processmaybemeasured,aswithShannon’smodel,asthelogarithm[Har28]of theinverseof theprobabilityof thestateof naturefoundat theoutputof theprocess.Thismaybeusedfor thevaluesof bothdiscreteandcontinuousoutputvariables.An outputvariablewith a valuehaving associatedprobabilityof

�����maybecomputedashaving ��� � �����

bitsof information,for example.

Wereferto aprocessthatproducesoutputfrom acontext (definedasthesetofvaluespresentedto theinputof aprocess)asaninformationchannel.Thechannelis thatsetof componentsthatimplementsthefunctionalityinherentin theprocess.A channelmightbeimplementedby acomputerandits program,or it mightbeamechanicaldevicesuchasapantographthatreproducesaninputmovementat theoutputwith a pre-specifieddegreeof magnification.

Processesmayconsistentlyproducethesameresultwhengiventhesameinputandcontext. Thesedeterministicprocessesarewhatmostpeoplemeanwhentheyrefer to a processin anunqualifiedmanner. Giventhesameinput, two identicalprobabilisticprocessesmayproducedifferentresults.Theoutputof a probabilis-tic processmay be describedby a probabilisticdistribution or densityfunction.Observersof suchaprocessmayunderstandtherandomnessascomingfrom twosources:inherentlyrandomprocessesanderror-producingnoise.

5

The processbaseddefinition of informationdiffers in several respectsfromShannon’s modelof information,to the extent thatShannon’s ideasaboutinfor-mationmaybeseparatedfrom thosehehadaboutcommunication.Shannonviewscommunicationsystemsprimarily astransmittingsymbolsbetweenasourceandadestination.This is very differentfrom theprocessmodelproposedabove whichproducesoutput from input, with no explicit attemptbeingmadeto encodeorrepresentsomethingsymbolically, or to be messagebased.Shannonnotesthathis theory is “quite different from classicalcommunicationengineeringtheorywhichdealswith thedevicesemployedbut notwith thatwhich is communicated”[Sha93a,p. 212]. Shannon’s focusclearlywason themessage.

UnlikeShannon’s model,theprocessmodeldescribesinformationastheout-comeof any process,not just anencodingprocessin a symbolbasedcommuni-cationsystem. The processmodelof information,for example,canbe usedtodescribeanadditionprocessthatacceptstwo inputsandproducesthesum(infor-mation)at the output. This outputis informative aboutthe additive processandthe inputs. This notionof aboutnessis similar to Devlin’s notionof a constraint[Dev91]. While theadditionprocesscouldbe interpretedasencodingthe input,that is certainlynot a very naturalinterpretationand is certainlynot a requiredinterpretationif we wish to understandarithmeticsums. Similarly, the processmodelof informationmeasuresthe informationat the outputasproportionaltothenumberof possiblestatesandtheir relative frequency in theoutput.

Eachprocessacceptsinputs or environmentalcharacteristicsand producessomething(or nothing),giventheseinput values.Thefunction ������� denotestheprocessing(or encoding)of � by function ������� A function ����� , whencombinedwith its functionalinverse��� �!��� , acceptsa variable � andproducesasoutput � ,which waspresentedto the processesat the input; i.e. �"���#� �!�����$� � ��� For ex-ample,the squareandthe square-rootfunctionsarewell known functionswitheachbeing the inverseof the other for positive numbers. Thus, % & � � & and' % &�( � � &)�

Consideragaintheadditionfunctionandits inverse.An additionfunctionthatacceptstwo inputs and producesan output, the sum, discardsinformation thatwaspresentat the input (theoutputof otherprocesses).Onecannotwork back-wardsfrom only thesumto theoriginal two numbers(thereaderis encouragedtoconsiderwhattheoriginalpairof numbersmighthavebeenif thesumis & .)

Considera functionwhich producesboth thesumof two presentednumbersandthedifferencebetweenthetwo numbers.This functionhasn’t discardedanyinformationin producingthesumandthedifference,in thatonecanwork back-wardto producetheoriginalnumbers.Considera sumof 3 anda differenceof 1;onecansolve �+*-, � & and �/.0, �1�

to produce, �2�and � �3� � Givena

losslessprocess,in whichnoinformationis lostduringprocessing,thereis alwaysaninverseprocessthatcanrecreatetheinput to thefirst process.

Is thisprecisemodelof informationtoonarrow to beof muchpracticaluseby

6

scholars?Saracevic andKantor [SK97] suggestthatsuchnormative approaches,including “formal and rigorousmodelsinvolving uncertainty....” (p. 532) take“the narrowestview of information.” We disagree,believing that the above rig-orousandgeneralmodelrepresentsa very broadview of information,with othermodelsof informationbeingcapableof beingshown to be specialcasesof thismodel. Below, we presentwhatwe similarly considerto bea general,yet rigor-ous,definitionof communication.

3 Defining Communication

Hundredsof explicit andimplicit definitionsof communicationhave beenpub-lishedin thecommunicationandrelatedliteraturesfor useby scholarsandprac-titionerstrying to describe,predict,andunderstandcommunicative phenomena.Thesedefinitionsvary aroundthe commonlanguagedefinitions,with variationsdependingon individual scholarlyinterestsandgeneralscholarlytrends.Thedi-versedefinitionsof communicationofferedby Hauser[Hau96, p. 7] serve asarepresentative, albeit small, sampleof ideasaboutcommunicationfrom a widerangeof disciplines. Of seven definitionsprovided by Hauser, threedefinitionsof communicationplacecommunicationin thecontext of humansor organisms,while amajoritymentiontheeffectof amessageon its recipient.

Edward Wilson notesthat “the ongoingfragmentationof knowledgeandre-sulting chaosin philosophyarenot reflectionsof the real world but artifactsofscholarship”[Wil98, p. 8]. Thedefinitionof communicationdevelopedbelow isbothrigorousandgeneralin capturingall andonly thecommunicationphenomenain the “real world.” We arenot trying to build on traditionaldefinitionsof com-municationandour definitionisn’t “an artifactof scholarship”;instead,we builda modelof communicationson botha precisedefinitionof informationandon alist of requiredcharacteristicsfor adefinitionof communication.Thus,thereis noexplicit preferencefor which sideshouldwin in long-runningintra-disciplinarydebates,suchaswhetherthereis intrapersonalcommunication,for example.Weassumethatcommunicationhasthefollowingcharacteristics(derivedin partfromquestionsposedby Motley [Mot90]):

1. communicationis characterizedby informationtransfer,

2. processingtakesplacein communicationsystems,

3. both the senderandthe receiver areactively involved in a communicationsystem,and

4. thequalityof communicationsvaries.

7

Communicationshouldbe definedwithout regardto possiblyfalseassumptionsmadeaboutportionsof the communicationsystem.Put differently, we will as-sumethat numbers1 through4 above needbe the only characteristicsusedindevelopinga definitionof communication.We thusbegin developingour modelof communicationfrom observablephenomenaandthe desirablecharacteristicsof adefinitiondescribingthephenomena,not from traditionaldefinitions.

Wechoosenot to placeanemphasisoncommunicationassymbolicin nature.Motley [Mot90] emphasizesthis symbolic nature,mentioningthat “Cronkhite(1986) presentsa very compellingcasefor symbolic behavior as the commondenominatorof all communicationstudy.” However, asHauser[Hau96,p. 507]notes,thesymbolicapproachmaynot bethebestin all situations,althoughit iscertainlya reasonableassumptionfor somemodelsof communication.We alsochoosenot to focuson intentionality, motivation,or thebehavior of thesenderorreceiver. Instead,we adoptanapproachthat,with theprecisemodelof informa-tion developedabove, providesuswith a morephysical,observable,andprecisesetof requirementsfor a definitionof communication.

We may definecommunicationin a preciseand information-basedmannerusingthecharacteristicsabove,allowing usto understandthemechanismsunder-lying communication.

Communicationoccursif, andonly if, informationmovesfrom theinput to oneprocessto the outputfrom a secondprocess,the latterprocessbeingtheinverseof thefirst process.

We refer to the informationat theoutputof this inverse,receiving, process,asacommunication.Communicationis morecomplex thaninformation;communica-tion processesarecomposedof multiplecomplementaryinformativeprocesses.

Herewe have two informativeprocesses,thesecondof which “undoes”whatthefirst process“does.” Viewedloosely, hearing,for example,undoeswhatspeak-ing does.Telephonesprovide communicationcircuits,providing aninput deviceat one end of a connectionand an inverse,decodingprocessat the other end.Similarly, the languagecomponentof a persontalking on the telephonemay besaidto communicatewith the(inverse)languagecomponentof the listener. Theknowledgecomponentsof thetwo arein communication.

Usingthismodelof communication,wemaydefineacommunicationreceiverastheimplementationof a function � � � �4� where ����� is referredto asthecommu-nicationtransmitter.

If thefirst processmerelycopiestheinput to theoutput,andtheinversepro-cesscopiesits inputto its output,communicationis takingplaceunderourmodel.This “straightwire” systemclearlyneitherencodesnor decodes,showing anob-vious differencebetweenthis model and other modelsbasedon encodinganddecodingof messages. Noneof thesethreeconceptsis essentialto our modelofcommunication.Weusethecommunicationjargon“encoding”and“decoding”to

8

Knowledge 5 Knowledge6 7Language 5 Language6 7

Sound 8+9 Sound

Figure1: A hierarchicalmodelof humancommunicationsrepresentingthe pas-sageof somethingbeingtransmitted,beingtransformedor encoded(left) andde-coded(right). Thestraightright arrow representsa physicalconnectionbetweentheprocesses,while thesquiglyright arrows for thehigherlevelsrepresentchan-nelsprovidedby thelower level processes.

represent����� and ��� �!��� but notethatin casessuchasthe“straightwire” systemitbecomesclearthatthisterminologycapturestheessenceof thiscopying operationonly weakly, if atall.

Not all informationtransmittedrepresentscommunication.Givencommuni-cationdefinedin termsof inverseprocesses,thepageyou arereadingisn’t com-municatingwith you. You arereceiving the informationthat is on the pagebe-causeof visual processes.The authoris communicatingwith you throughpro-cessesthatfirst took ideasresultingin written text, andaninverseprocesswithinthereaderis takingwritten text andtransformingit backinto thoughts.Similarly,if onepersonis talking to anotherandis nervous,thenervousnessmaybecom-municatedto anyonewho cantranslateobservedperspiringor a quiveringvoiceor shakinghandsinto anunderstandingthatthefirst personis nervous.

Figure1showsahierarchyof processesusedfor informationtransferandcom-munication.Considereachhierarchyrepresentingan individual. Onecommuni-catesheror his knowledgein language,which is furthertransformedor encodedassounds.Thesearedecodedontheright handside,producinglanguagefrom thesoundsusinganinversefunction,andthelanguageis convertedbackinto knowl-edge,againthroughthefunctioninverseto thefunctionthatinitially transformedtheinformation.Thedecodingtakesplaceon theright handside,with ascendingfunction ��� �!����� representingtheinverseof the ���:��� function;thefunctionsontheright handsideof Figure1 “undo” whattheearlierprocesseson theleft “did.”

In additionto differentphysicalphenomenabeingmodeledby specificlayersof ahierarchy, psychologicalandepistemologicalphenomenamayberepresentedby layersin acommunicationhierarchy[Los97]. For example,aperceptionfunc-tion mayhaveasoutputabelief instilled in theindividual,andabelief functionin

9

turnmayproduceknowledge.Notethat thechoiceof a hierarchyasa structureis arbitraryandthata setof

communicationprocessesmaybealsounderstoodasplacedin astraightline,withonesetof processesproducinginformationwhich is thenpresentedto theinputsto theinverseprocesses.Usingthenotionof ahierarchyfacilitatesanunderstand-ing of the relationshipbetweenrelatedindividual informationprocessesandthelargercommunicationprocesses.We alsodo not meanto imply thatencodingordecodingasShannonviewedtheseoperationsareessentialto this modelof com-munication.While thehierarchicalmodelemphasizestransformationswhichmaybeinterpretedasthetraditionalencodinganddecodingfoundin moretraditionalShannesquemodels,rememberthatastraightwire mayallow oneendto commu-nicatewith the otherend,with little processingpresentto suggestthe encodinganddecodingmetaphor.

This modelof communicationdiffersfrom many moretraditionalapproachesto communication.At thecoreof thedifferenceis thatthemodelusedhereis nothuman-centered.It doesn’t have beinghuman,or evenorganic,asa prerequisitefor beinganendpointin a communicationsystem.This expandscommunicationsystemsto includeprocessorssuchascomputers,aswell aslesssophisticatedre-producingdevicessuchasphotocopiers.By not arbitrarily limiting thesystemtohumans,little is lost andmuchis gained,bothby describingphenomenaoutsidehumansthatseemvery similar to what is traditionallyreferredto ascommunica-tion andby greatlysimplifying thedefinitionandmakingit precise.Becausethedefinitionis not explicitly biological in nature,it is alsonot explicitly intentionalin nature.Intentionalcharacteristicscanbeincorporatedinto processes;modelingcommunicationas complementaryprocessescan be consistentwith intentionalprocessesandhumancommunication,allowing modelsof communicationsuchasthat describedby SperberandWilson [SW95] to be seenasdescribingcom-municationsthatarespecialcasesof this moregeneralmodelof communication.Unlike many modelsof communication,however, our communicationprocessesarenot limited to intentionalor humanphenomena.Similarly, communicationdoesnot have to transmitmeaningor be intrinsically symbolic,althoughthesephenomenacanbe incorporated.This differs from Shannon’s modelof commu-nication,which beginswith a sourcewhich ”producesa messageor sequenceofmessagesto be communicatedto the receiving terminal” [Sha93b,p. 6]. Simi-larly, whenShannonsaysthatwecan”roughly classifycommunicationsystems,”thecategoriesaredescribedashaving amessagecomponent[Sha93b,p. 7–8].

ScholarssuchasOliphant[Oli97] haveacceptedthatmany of thepopularex-istingdefinitionsof informationareinadequatefor theirwork andhavedevelopedtheir own definitionsfor thepurposesof their research.We believe that thepro-poseddefinitionof communicationis a universal,domainindependentdefinitionthatallows for many of theconcernsmotivatingotherdefinitionsof information.We thusfeel thatthisdefinitionservesasa commondefinitionof communication

10

for thefield, providing acommonlanguage.Additionalconstraintsmaybeaddedto this,but they arenotpartof thefundamentalnatureof communication;they areadditionalconstraintsthatmayor maynot beworthyof discussionbasedon theirown merits.

As wedefinedaninformationchannel,wemaydefineacommunicationchan-nel as the set of componentsin the universethat implementthe functionalityneededfor thecommunicationprocessto take place.Becausecommunicationre-quirestwo or moreinformationprocesses,a communicationchannelalwayscon-sistsof two or more informationchannelsin series. By usingthe word needed,we refer only to thosecomponentsthat directly andobviously contribute to theoperationof the channel,ignoring the fact that a butterfly moving on the othersideof theworld doesaffect theperformanceof a communicationsystemon thereader’s sideof theworld.

Using this model of communication,we may measureboth the amountofinformationthatis presentat differentpointsin themodel.For example,we maymeasurethe amountof informationenteringthe systemaswell ascaptureanddescribethe informationat the output of the first and secondprocessesand atintermediatepointsin acommunicationhierarchy.

4 Basisfor a Hierar chy

For a systemto be a communicationsystem,it mustbe composedof two com-plementaryinformativeprocesses.If we areto studya communicationsystemascomposedof morethanonefunction, that is, we chooseto treatit assomethingother thanan indivisible whole, it is necessaryto breakthe larger processintosmallerprocesses.Centralto breakinga processinto partsis determiningwhichdescriptionsof processesaremostnaturalandbesthelpusdescribe,predict,andunderstandthe communicationsystem. In somedecompositions,the resultingsub-processesmay appearto be natural,having a reasonableexplanation,whilein othercases,thedivision into sub-processeswill appearveryarbitraryin naturewith unnaturalsub-processes.Obviously, a decompositionthatproducesreason-ableexplanationsis superiorto onewheretheboundariesbetweenprocessesarearbitrarilychosen.

Communicationprocessesmay be delimitedso that they representan entirereceiveror anentiresender, or processesmaybeviewedasbeingmuchsmaller, sothatmany of thesesmallerprocessestogetherconstitutetheprocessingcapabilityof an individual. If the mostnaturalform of a hierarchyis with a largenumberof smallprocesses,it maybeeasiestfor anorganismto learnthedetailsof eachsmallerfunctioninsteadof learningthenatureof a single,largefunction. On theotherhand,in somecasesit may be beneficialto definea processlarger thanasingleindividual,with, for example,a groupof peopleproducinga document,oran individual readinga document,includingthedocumentandits production,in

11

asinglelargeprocess.Therearea numberof processesthatoccurfrequentlyin higherlevel species,

including a numberof componentsto naturallanguageprocessing,suchas themovementof vocal cords,the receptionof soundandits translationinto neuralsignals,syntacticprocessing,andphonologicalprocessing.Hocketthassuggestedthatthereareat least13differentfeatures(andthusfunctions)in humancommu-nication [Hau96,p. 47]. No other specieshasall 13 features,althoughmanyspecieshave several. Thesemayserve asa basisfor anunderstandingof thehu-manprocessesin naturallanguagecommunicationhierarchies.

Thoseprocessesthat exist andare observableare more likely to have con-tributed to the survival of the greatersystemswithin which the processexiststhanarelessadaptableprocessesthatdid not contributeasmuch.A processmaycontribute to its own survival, or it may contribute to the survival of similar oridenticalprocessesin othersystems,whencontributing to the survival of othersincreasesthe chancefor similar processesto appearin future systems.For ex-ample,someonewho cancommunicatethe natureof a vaccinefor a childhooddiseasethatcansave thelivesof millions of childrenmaynot increasetheir ownchanceof survival but will increasethe numberof communicative humanswithintellectualabilitiessimilar in somerespectsto thoseof thevaccine’sdeveloper.

We refer to systemsthat take on increasinglysophisticatedandnon-randomstructuresasthey evolve asself-organizingsystems.Functionsmay be learnedor mayevolve throughself-organizationover time. This oftenoccursbecausein-creasedorganizationor structureitself leadsto increasedadaptabilityandsurvival,althoughwe shouldnot make themistake of assumingthat increasedcomplexitynecessarilyimpliesincreasedsurvivability [Gou97]. Throughlearningprocesses,a function may infer the characteristicsand parametersof a function of inter-estthroughsupervisedlearning,wherelabeledcasesarelearnedwheninferringa generalizingprinciple. The effect of learningalsomay be producedthroughevolution, with randomlygeneratedvariationseithersurviving with increasedordecreasedfrequency over time. Thishastheeffectof learningadaptivecharacter-istics.

Learningcommunicative functionsthroughevolutionary processesrequiresthepresenceof a higherlevel layer thatbothproducesandreceivesinformation.In thecaseof beessaidto becommunicatingthroughdancing, thereis a processabovethedancingprocess(whichhasevolved)thatallows thedancingprocesstobebeevolutionarilysupportedby increasingthesurvival rateof beeswith dancing(or understanding)skills.

Any processlocatedaboveanotherprocessmaybeagoalprocessthatrewards,allowing the functionbeforeit to evolve. We refer to a function thatmaximizesa goal function asmaximallyuseful. For an informative function to be learnedgenetically, it mustsupportagoalwhich is relatively constantin thespecies,withevolution leadingtoward it becomingmaximally useful. Goal processesoften

12

contribute towardreproductive behavior or aresurvival related.Communicationprocessesproduceresultswith thesystemsin which they occursuchthatsimilarprocessesareas likely or more likely to survive andreproducethan they werebefore.This is becauseintroducingcommunicativecapabilitiesessentiallymakesa systemlarger, which canleadto increasedself-organization,thanwould bethecasewith individual,smaller, non-communicativesystems.

Givenourmodelof communication,communicationsystemsarelikely to de-velopin self-organizingsystemsbecauseof their supportfor whatareconceptu-ally largersystems,leadingto moreself-organizationandincreasedadaptability.We assumethat it is almostaslikely that if oneprocessevolvesthenits inversewill alsoevolve. We alsoassumethat informationaboutanothersystemmaybebeneficialto thesystemin questionundermany circumstancesandthuscommu-nicationsystemsareevolutionarilyadaptive.

5 The Performanceof Processes

All receiving informativeprocesseswithin acommunicationsystemmaybeviewedasfiltering processes,selectively providing informationto a processimmediatelysucceeding(above) it in the right half of the hierarchy. Someprocessespassnoinformationto succeedingneighboringprocesses,someprocessesselectively passinformation,while otherspassall input informationto theirneighbor.

A neighborprocesswhoseoutputis inputtoasecondprocessis referredtoasacontext for thesecondprocess.Whenall input to aparticularprocesscomesfromtheoutputof anotherprocess,thelatterprocessis the full or completecontext fortheformerprocess.Thecontext maybeviewedasproviding aqueryor statementof informationneedfor afiltering process.

A filtering processacceptsinformation from the processbelow and, basedupon the context and the interestsandneedsit represents,will transmitor nottransmitinformationto thelevel aboveit, dependingontheincominginformationandthe context provided from its predecessorandotherprocesses.Informationthatis passedto ahigherlevel processmaymodify thecontext itself andmaythusaffect futurefiltering actions.Thefiltering processmaythusberecursive.

Considera messagewith one binary characteristicwhich would occur in anoiselessinformative processtransmittinga singlebit. Thecharacteristicoccurswith probability(andaveragefrequency) ; in messagesto bepassed(referredtoasrelevant messages)andprobability (andaveragefrequency) < in all messages[Los98]. If a filter attemptsto passonly thosemessagesat or above theaveragepositionof relevant messagesin a list of messagesorderedby decreasingrele-vance,thepercentof all messagesthatwill bepassedis

� .=� � .";>*/<$� � � � Smallervaluesrepresentbetterfiltering performance.Clearly, otherfiltering performancemeasureswill beneededwith morecomplex termrelationshipsandprobabilisticdistributions,but this indicatesthekind of resultthatmaybeobtained,increasing

13

ourunderstandingof communicationsystems.

6 Imperfect Communication

Defining communicationas the outputof two complementaryinformative pro-cessesdescribescommunicationin anidealnoiselessenvironmentwherethesec-ondfunctionis a perfectinverseof theother. Here,thesecondfunctionperfectly“undoes”what thefirst function“does” andthereis no noiseintroducedinto thesystem.

In therealworld,however, thereis almostalwaysnoisepresentandcommuni-cationis imperfect,exceptin thepresenceof errorcorrectingcodesthatallow therecipientof amessageotherthanthemessagethatwassentto recovertheoriginalmessage[Ham86,Los90].

We refer to the outputof a noisy communicationchannelasnoisemodifiedcommunication. Communicationsreceived that have beennoisemodified butcanbeunderstoodaccuratelyor usedby a human,evengiventhenoiseinducedchangesin it, areacceptablenoisycommunications. Theseerrorsor noisemaybeinducedby externalprocessesandevents,causingtheinformativeprocessestofunctiondifferentlythanthey wouldwithouttheerrorproducingprocess.Noiseorerrorsin theinformationmaybeviewedasadditive if theoperationsof eachpro-cessareindependentof theoperationsof theotherprocessesandtheseprocessesarein series.

In many cases,theinversionprovidedby a process� or � � � is imperfect,thatis, for some ��? it will be the casethat �"�4� � � �����$�A@� ��� For example,considerthe casewherethe authorwho grew up in the northernUnited Statesreferstosnowwhentalking with his daughter, who hasspenther life in thesouthernhalfof thecountry. Theauthor’s hierarchyencodesa certainsetof meaningsassnowwhile hisdaughterdecodesthetermsnowasaratherdifferentphenomenon.Herethedaughter’s receiving function isn’t theexact inverseof her father’s encodingfunction.

The acceptabilityof noise-modifiedor imperfectly invertedcommunicationmaybebestviewedaseitherbinaryor continuous.Theexactnatureof theaccept-ability functionis anempiricalquestionto bedeterminedfor any givencommuni-cationchannel.Binary acceptabilityoccurswhena messageis eitheracceptableor notacceptable.Acceptabilitymaythusoccurwhenthemessageis similar to atleastacertaindegreeto anexpectedmessage.Acceptabilitymayalsobecontinu-ouslyvalued,thatis, theacceptabilityitself cantakeany of arangeof valuesfrom,for example,0 to 1, dependingon how closetheacceptabilityis to thebest-caseacceptabilityandtheworst-caseacceptability.

14

7 Branching Communication Hierar chies

Theproposeddefinitionof communicationsystemsascomplementaryinformativeprocessesenablesoneto accuratelymodelandunderstandarangeof communica-tion phenomena.With thismodel,wecandescribeandpredictthecharacteristicsof previouslyunstudiedsystems.While thelinearhierarchypresentedabovecap-turessimplecommunicationenvironments,the usefulnessof the modelmay beimprovedsignificantlyif weallow asingleprocessto havemultipleprocessesbe-neathit (whichwereferto asmultiple feet) andmultipleprocessesaboveasingleprocess(which we refer to asmultiple heads). Doing this allows claims to bemadeaboutmultiple headsandfeet that morecloselyreflect the complexity ofreal world settings.In addition,graphtheoreticcommunicationmodelsmay beappliedto complex structures,providing a wider rangeof both quantitative andqualitativemodelsof communication.

Considerthat the authormight be able to communicatewith you throughthispublication,or throughemail,hand-writtencorrespondence,or by telephone.Eachdifferentmediumcanbe representedby a differentfoot. Theauthor’s lan-guagecomponentcanbe understoodashaving below it several feet, eachcon-sistingof oneor morecommunicationprocessesconnectingthe authorandthereader.

Modeling socialcommunicationsystemsmay be facilitatedby usingmulti-ple “feet” in the model. A group of peoplehaving a particularrelationshipisreferredto asa socialnetwork, which may be studiedqualitatively [Cha92]orquantitatively [WF94]. Thestudyof socialnetworkscanbeviewedasthestudyof which processingfeet exist for eachindividual and what other feet in whatotherhierarchiesindividualsusefor communication.For example,assumethateachindividual hasa hierarchyandthat therearethreelevelson eachhierarchy:high (H), medium(M), andlow(L):

HBMBL

Considerwherea particularsocialnetwork is implementedby all theL levelsforthe individualsin the network beingconnectedbecauseall the L levels are theidenticaloutputsof a single larger processwhich acceptsas input informationprovidedby any of theindividualsin thegroup.Wheneachindividual in a setofindividualsis connectedandthereis perfectcommunication,thenwe may referto this setof peopleasconstitutinga socialnetwork. Eachpersonin this group

15

knowledgeC DSpanish FrenchD C

visionC Demail bookE E

Figure2: The hierarchiesfor incoming information from two differentwrittensourcesandin two differentlanguages.

receivesinformationat theL level thatindividualsoutsidethenetwork don’t nec-essarilyreceive.

Thediffusionof informationor of aninnovationis the transmissionof infor-mationto someor all of themembersof a domainof individuals,with thecom-municationchannelsgoing from oneindividual to oneor moreothers,with therecipientin turn possiblytransmittingto oneor moreothers,with time delaysof-tenoccurringbetweentransmissions[MP85, Rog83]. Thediffusing informationmayspreadfrom onepersonto anotherthroughany of a numberof communica-tion mediaandatvaryingrates[Cha86, Jen82, LB75, MP85, Rog83].

Personto persondiffusionis modeledby eachpersonhaving a foot connectedto several otherpeople’s feet, eachconnectionrepresentinga possiblepoint ofcontactbetweenthis individualandanother. Diffusionthroughanindividual thusdependson theindividual receiving theinformationandthedecisionbeingmadeto passalongtheinformation.

Multiple headsmay occurwithin the processingof informationby an indi-vidualhuman.InformationthatI receive maybeprocessedin differentwaysandviewed differently by the author, who is a male,a father, anda teacher, all ofwhich mayaffect differentwaysthe informationis processedgiventhedifferentroles.Thesedifferentwaysof knowing allow oneto communicatewith someoneelsewith thesamewayof knowing,while communicationwith someoneelsewillresultin inverse-functionnoiseaffectingwhatis received.

Considerasituationwherewehavetwooverlappinggroupsof languagespeak-ers,with onegroupspeakingFrenchandonegroupspeakingSpanish.Thereisalsoa groupof peopleusingemail,anda groupnot usingemail. We maymodelthissituationasin Figure2whichshowsmulti-headedandmulti-footedprocesses.

16

Thereare consequencesto modelinga communicationsystemas a multi-headedmulti-footedmonster. Sucha modelclaimsthathierarchiesof processes“meet” andthat thereis a commonmeetingpoint that is perhapscritical to theeffectiveoperationof thecommunicationsystem.In addition,meetingpointsthatbracket a setof processesabove andbelow suggeststhateachof theseprocessesmay be treatedasa singletype of functionalunit. The existenceof theseunitshasanaturalsupportandthestudyof theseunitsmaybemoreprofitablethanthestudyof units with lesserdegreesof support. Eachunit may evolve, increasingtherateof survival andreproductionof thosecreatureswith this function.

8 Characterizing CommunicativeProcesses

Being able to preciselycharacterizea particularcommunicationsystemallowsoneto comparecommunicationsystems,includingdecidingwhethertwo systemsareequivalent,aswell asproviding amethodby whichonecanrankcommunica-tion systemsbasedon their characterization.Onemayalsocharacterizea systemby namingor describingit, baseduponits content. The Minimum DescriptionLength(MDL) for a systemis theshortestlengththatcandescribeanddifferen-tiateoneprocessfrom othersin thedomainof possibleprocesses[Ris89,WF87].TheMinimal Description(MD) characterizeswhatmakesup a system,andthusidentifieswhat is complex and specialabouteachsystem. A systemwhich iseasyto describehasa relatively shortdescriptionanda smallMDL, while a verycomplex system,approachingrandomness,wouldhavea relatively largeMDL.

We aredescribingonly the two informative processesdefiningthe commu-nicationsystem. If we have two processes,andtherearemany otherprocessesbelow them on the hierarchy, we may computethe MDL by consideringonlythecomplexity of the two informative processes,with no attentionbeingpaid toprocessesbelow. We might, however, chooseto view the world as containinglarger processes,e.g.,you asa processandme asa process,in which casetheMDL would becomputedfor meor for you andmight or might not includethismedium.Becausethereaderdiffers from theauthor, thereaderfunction is not aperfectinverseof the writer function, makingthe MDL muchlarger thanif theauthorwerewriting for himself,thereaderthenbeinganear-perfectinverseof thewriter. Theorderingof systems,from simpleto complex, canbeaccomplishedbylisting thesystemsin orderof theirMDL value,e.g.

Order MinimumDescriptionLength(MDL) in bits1

�GF�HI�J�(simple)

2�GF � �J�GF F

(mediumcomplexity)3

�GF�KI�J�GF F�F F F(verycomplex)

An orderingsuchasthismaybeusedto comparecommunicationprocesses.

17

9 Summary and Conclusions

A satisfactorydefinition of communicationshoulddescribethe phenomenonofinterest.A bettermodelwill bothdescribeexistingeventsandpredictthecharac-teristicsof futurecommunications.Thebestdefinitionswould helpexplain whatis occurring,aswell asallow usto describeandpredict.Themodelof communi-cationascomplementaryinformativeprocessesis basedon a setof requirementsfor adefinitionof communicationandaprecisedefinitionof information.Thepre-cisionin this modelof information,combinedwith theprecisionof thedefinitionof communication,allowsusto stateexplicitly whatis andwhatisn’t informationandwhatis andwhatisn’t communication.Onemaythendefinea communicationaswhatis transmittedfrom thebeginningof oneprocessto theoutputof aprocesswith theinversefunctionalityof thefirst process.

Most phenomenareferredto in the form “X to X communication,” suchas“humanto humancommunication”or “machineto machinecommunication,” arecommunicationunderour definitionof communication.Theseexpressionsimplycommunicationbetweenthesamelevelsof two differenthierarchies,or, morefor-mally, betweena function ���4� andits inverse��� �!���L� Definingcommunicationastwo complementaryprocessesallows us to bothpredictwhatwill happenin thepresenceof noiseor thefailureof functionsto beexactinverses.Onecanalsoun-derstandthehumanaspectsof communication,aswell asthelow level biologicalandphysicalphenomenasuchasspeechor hearingandthephysicaltransmissionof soundthroughtheatmosphereor thetransmissionof wordsthroughthemove-mentof writtenpages.

Our precisedefinitionallows us to includephenomenaaswell asto excludephenomena.A definition(suchasours)of a phenomenoncentralto a disciplinesuchascommunicationmaydiminishor excludesomephenomenain two ways:by sayingthatsomethingisn’t theconceptbeingdefined,or thata characteristicisn’t an intrinsic partof thephenomenabeingdefined,decreasingits importanceto thedefinitionandto thefield, while not excludingit. A precisedefinitionmaynot explicitly includea numberof phenomenaof potentialinterestto communi-cationscholars,while beingconsistentwith lessinclusive definitionsoftenusedin the field of Communications.Unlike someotherdefinitions,our modelsug-geststhata wide rangeof phenomenacanbeviewedaspartof a communicativesystem,but arenot necessarilypart of communication.For example,given thecharacteristicsof communicationdescribedabove, this pagedoesnot communi-catewith thereader, but we might saythat theauthoris communicatingwith thereader. Characteristicsof communication,suchasintention,aren’t includeddi-rectly in theproposeddefinition; it isn’t excluded,but it isn’t explicitly includedeither. Thedefinitiondiscussedhereis moreinclusivethanmostotherdefinitions,while at the sametime having morerigor thanmost. This increasesour under-standingof communication,andcanleadto a morestablefoundationfor theart

18

andscienceunderlyingtheinformationandcommunicationprofessions,includinglibrarianship.

References[BH55] YehoshuaBar-Hillel. An examinationof informationtheory. Philosophyof Science,

22:86–105,1955.

[BHC53] YehoshuaBar-Hillel andRudolf Carnap.Semanticinformation. TheBritish Journalfor thePhilosophyof Science, 4(13):147–157,August1953.

[Bri56] LeonBrillouin. ScienceandInformationTheory. AcademicPress,New York, 1956.

[Cha77] G. J.Chaitin.Algorithmic informationtheory. IBM Journalof Research andDevelop-ment, 21(4):350–359,July 1977.

[Cha86] ElfredaA. Chatman.Diffusiontheory:A review andtestof a conceptualmodelin in-formationdiffusion.Journalof theAmericanSocietyfor InformationScience, 37:377–386,November1986.

[Cha92] ElfredaA. Chatman. TheInformationWorld of RetriedWomen. Greenwood Press,New York, 1992.

[Der93] BrendaDervin. Verbingcommunication:Mandatefor disciplinaryinvention.Journalof Communication, 43(3):45–54,Summer1993.

[Dev91] KeithDevlin. Logic andInformation. CambridgeUniversityPress,Cambridge,1991.

[Dre81] FredI. Dretske.Knowledge& theFlow of Information. MIT Press,Cambridge,Mass.,1981.

[Fis25] RonaldA. Fisher. Theoryof statisticalestimation. Proceedingsof the CambridgePhilosophicalSociety, 22(5):700–725,1925.

[Gou97] StephenJ.Gould. Full House:TheSpreadof ExcellencefromPlato to Darwin. Ran-domHouse,New York, 1997.

[Ham86] RichardHamming.CodingandInformationTheory. Prentice-Hall,EnglewoodCliffs,N.J.,secondedition,1986.

[Har28] R. V. L. Hartley. Transmissionof information.Bell SystemTechnical Journal, 7:535–563,1928.

[Hau96] MarcD. Hauser. TheEvolutionof Communication. MIT Press,Cambridge,MA, 1996.

[Jen82] RichardJensen.Adoption and diffusion of an innovation of uncertainprofitability.Journalof EconomicTheory, 27(1):182–193,1982.

[KMA82] A. J. Kfoury, RobertN. Moll, andMichael A. Arbib. A ProgrammingApproach toComputability. Springer-Verlag,New York, 1982.

[Kol65] Andrei Nikolayevich Kolmogorov. Threeapproachesto thequantitative definitionofinformation.ProblemyPeredachi Informatsii, 1(1):3–11,1965.

[LB75] NanLin andRonaldS.Burt. Differentialeffectsof informationchannelsin theprocessof innovationdiffusion.SocialForces, 54(1):256–274,September1975.

[Los90] RobertM. Losee.TheScienceof Information: MeasurementandApplications. Aca-demicPress,SanDiego,1990.

19

[Los97] RobertM. Losee. A disciplineindependentdefinitionof information. Journal of theAmericanSocietyfor InformationScience, 48(3):254–269,March1997.

[Los98] RobertM. Losee. Text Retrieval and Filtering: Analytic Modelsof Performance.Kluwer, Boston,1998.

[Mac69] DonaldM. MacKay. Information,MechanismandMeaning. MIT Press,Cambridge,Mass.,1969.

[Mot90] MichaelT. Motley. On whetheronecan(not)not communicate:An examinationviatraditional communicationpostulates. WesternJournal of Speech Communication,54:1–20,Winter1990.

[MP85] Vijay MahajanandRobertA. Peterson.Modelsfor InnovationDiffusion. SagePubli-cations,Beverly Hills, 1985.

[Oli97] MichaelOliphant. Formal Approachesto InnateandLearnedComunication:LayingtheFoundationfor Language. PhDthesis,Universityof California,SanDiego,1997.

[Ris89] JormaRissanen.StochasticComplexity in StatisticalInquiry. World Scientific,Singa-pore,1989.

[Rit86] David Ritchie. ShannonandWeaver: Unraveling theparadoxof information. Com-municationResearch, 13(2):278–298,April 1986.

[Rit91] David L. Ritchie. Information. Sage,Newbury Park,CA, 1991.

[Rog83] EverettM. Rogers.Diffusionof Innovations. FreePress,New York, 3rdedition,1983.

[Sha93a] ClaudeElwood Shannon.Article on informationtheoryfor encyclopediabritannica,14thedition.In N. J.A. SloaneandAaronD. Wyner, editors,ClaudeElwoodShannon:CollectedPapers, pages212–220.IEEEPress,1993.

[Sha93b] ClaudeElwood Shannon.The mathematicaltheoryof communication. In N. J. A.Sloaneand Aaron D. Wyner, editors,ClaudeElwood Shannon: CollectedPapers,pages5–83.IEEEPress,1993.

[SK97] Tefko Saracevic andPaul B. Kantor. Studyingthe valueof library and informationservices.partI. establishinga theoreticalframework. Journalof theAmericanSocietyfor InformationScience, 48(6):527–542,June1997.

[Sol64a] R. J. Solomonoff. A formal theoryof inductive inference.part I. InformationandControl, 7(1):1–22,1964.

[Sol64b] R. J. Solomonoff. A formal theoryof inductive inference.part II. InformationandControl, 7(2):224–254,1964.

[SW95] DanSperberandDeirdreWilson. Relevance:CommunicationandCognition. Black-well, Oxford,secondedition,1995.

[WB68] C.S.WallaceandD. M. Boulton.An informationmeasurefor classification.ComputerJournal, 11:185–194,1968.

[WF87] C.S.WallaceandP. R.Freeman.Estimationandinferenceby compactcoding.Journalof theRoyalStatisticalSociety, B, 49(3):240–265,1987.

[WF94] Stanley Wassermanand KatherineFaust. SocialNetworkAnalysis. CambridgeU.Press,Cambridge,1994.

[Wil98] EdwardO. Wilson. Consilience. Knopf, New York, 1998.

20