Info introduction to [email protected] informatics.indiana.edu/rocha/academics/i501/...

[email protected]/rocha/academics/i501/

INDIANAUNIVERSITY

Informatics luis rocha 2017

I501introduction

to informatics

introduction to informaticslecture 5


INDIANAUNIVERSITY

I501introduction

to informatics


Readings until now

This week Young, T. (2017). “Cease and desist”. Nature, 541(7637), 430–

430. Lazebnik, Y [2002]. "Can a biologist fix a radio?--Or, what I

learned while studying apoptosis". Cancer Cell, 2(3):179-182. Optional

Presentations G.L. Ciampaglia, P. Shiralkar, L.M. Rocha, J. Bollen, F. Menczer,

A. Flammini [2015]. “Computational fact checking from knowledge networks.” PLoS One. 10(6): e0128193. Russo, Giampaolo

Piantadosi, S. T.,et al (2011). Word lengths are optimized for efficient communication. PNAS, 108(9), 3526–3529. Malic, Vincent

Lecture Notes The Nature of Information Formalizing and Modeling the World

Available http://canvas.iu.edu and listed at

http://informatics.indiana.edu/rocha/academics/i501 Also check out

Links and notes at http://sciber.blogspot.com/

introduction to informatics


INDIANAUNIVERSITY

I501introduction

to informatics


cybernetics

Postwar culture of problem solving Interdisciplinary teams Cross-disciplinary methodology Engineering-inspired Supremacy of mechanism

All can be axiomatized and computed Mculloch&Pitts’ work was major influence

“A logical calculus of the ideas immanent in nervous activity”. Bulletin of Mathematical Biophysics 5:115-133 (1943).

A Turing machine (any function) could be implemented with a network of simple binary switches (if circularity/feedback is present)

post-war science

Macy Conferences: 1946-53

Warren S. McCulloch

Claude ShannonMargaret Mead

Heinz Von Foerster


INDIANAUNIVERSITY


I501introduction

to informatics

design principles of computationBabbage/Lovelace, Turing’s tape, and roles of information

distinction between numbers that mean things and numbers that do things.


INDIANAUNIVERSITY

I501introduction

to informatics


At every discrete time instance the machine is in a single state

Program is a state transition table

state Read symbol

Next state

Write symbol

Tape move

0 4 1 - left

1 - 0 1 right


INDIANAUNIVERSITY


I501introduction

to informatics

Information as decrease in uncertainty . uncertainty-based information

C. E. Shannon, “A mathematical theory of communication”. Bell System Technical Journal, 27:379-423 and 623-656

Hartley, R.V.L., "Transmission of Information", Bell System Technical Journal, July 1928, p.535.

AAH 2log)( AAH 2log)(

Number of ChoicesMeasured in bits

i

n

iiS xpxpAH 2

1log)(

i

n

iiS xpxpAH 2

1log)(

Probability of alternativeMeasured in bits

information is surpriseinformation is surprise

including more structure reduces

surprise

including more structure reduces

surprise


INDIANAUNIVERSITY

I501introduction

to informatics


uncertainty

Mutual Information Amount of information about one variable that can be gained

(uncertainty reduced) by observing another variable Information Gain (Kullback-Leibler Divergence)

Difference between two probability distributions p and q, average number of bits per data point needed in order to represent q

(model approximation) as it deviates from p (“true” or theoretical distribution)

Other measures

n

i

m

j ji

jiji ypxp

yxpyxpYXI

1 12

,log,);(

n

i

m

j ji

jiji ypxp

yxpyxpYXI

1 12

,log,);(

),()()();( YXHYHXHYXI ),()()();( YXHYHXHYXI

n

i i

ii xq

xpxpXqXpIG1

2log))(),((

n

i i

ii xq

xpxpXqXpIG1

2log))(),((


INDIANAUNIVERSITY


I501introduction

to informatics

Do, do, do, do, do, do, do, do, doBefore we leave

Lemme tell y’all a lil’ somethingUptown Funk you up, Uptown Funk you up

Come on, danceJump on it

If you sexy, than flaunt itIf you freaky, than own it

Don’t brag about it, come show meCome on, dance

Jump on itIf you sexy, than flaunt it

Well, it’s Saturday night and we in the spot

Don’t believe me, just watchUptown Funk you up, Uptown Funk you up (say whaa?)

Uptown Funk you up, Uptown Funk you up

information of sequential messagesrate of removing uncertainty of each symbol

“syntactic” surprise But what about

function and meaning (semantics)?

“syntactic” surprise But what about

function and meaning (semantics)?


INDIANAUNIVERSITY

I501introduction

to informatics


Shannon’s mouse

trial and error algorithm information as reduction of uncertainty in the

presence of alternatives (combinatorics) lifelike behavior

trial and error to learn path from many alternatives adapts to new situations

how is learning achieved? Correct choices, information gained from reduced

uncertainty, must be stored in memory memory of information as a design principle

of intelligence in uncertain environments 75 bit memory stored in (telephone) switching relays

Brain as (switching) machine

controlling information to achieve life-like behavior


INDIANAUNIVERSITY

I501introduction

to informatics


formalizing knowledge Lord Kelvin's dictum

“When you can measure what you are speaking of and express it in numbers you know that on which you are discoursing. But if you cannot measure it and express it in numbers. your knowledge is of a very meagre and unsatisfactory kind.” 1824-1907

Absolute scale of temperature, underwater telegraph cables, thermodynamics

Physics The first science to construct precise, rigorous

formal theories of the world. relating the operation of rules upon symbols to the

law-like behavior of the World. Aristotle (384-322 BC) was first to relate symbols

more explicitly to the external world and to successively clarify the nature of the symbol-world relation.


INDIANAUNIVERSITY

I501introduction

to informatics


understanding Nature with symbols Aristotle (384-322 BC)

First to relate symbols more explicitly to the external world and to successively clarify the nature of the symbol-world relation. Student of Plato, educated Alexander the Great first to consider specific observable factors which

determine motion. In Physics

he recognized (mathematical) rules which could describe the relation between an object's weight, the medium's density and the consequent rate of motion (fall): (1) for freely falling or freely rising bodies, speed is

proportional to the density of the medium. (2) in forced motion, speed is proportional to the force

applied and inversely proportional to the mass of the body moved

first time that observable quantities had been expressed in symbolic (numerical) form allowing the results of observations to be used in calculations The nature of causation http://classics.mit.edu/Aristotle/physics.html

Raphael’s “Plato and Aristotle”


INDIANAUNIVERSITY

I501introduction

to informatics


abstracting relations Galileo (1564-1642)

Progressive dissociation of the symbols from objects The interrelationships among signs themselves

studied quite apart from the relations among the objects they represent Previously, symbols were still generally regarded as

inherent properties of the referent objects themselves Aristotle’s Physics postulated certain primary

qualities/elements such as “Fire”. Galileo regards "primary" properties as only those that can be mathematically quantified, such as size, shape and motion.

Newton (1643-1727) Extends process of abstraction

Distinguishes between symbols Arising from observation

represent initial conditions Arising from symbol relations

representing laws which govern the subsequent motion.


INDIANAUNIVERSITY

I501introduction

to informatics


Heinrich Hertz (1857-1894) Some facts about Hertz

First to broadcast and receive radio waves Established that light is a form of

electromagnetic radiation. His name is associated with the SI unit for

frequency Principles of Mechanics (1894)

Goal was to purge physics of mystical, undefined, unmeasured entities such as force (which one can infer but not

measure) Physical theories to be based only on

measurable quantities the results of measurements are symbols. Physical theory becomes about building

relationships among observationally-derived symbols: models what Hertz called "images."


INDIANAUNIVERSITY

I501introduction

to informatics


Using computation to model the World

World1

Measure

Symbols(Images)

Initial Conditions

Measure

Logical Consequence of Model

ModelFormal Rules

(syntax)

World2Physical Laws

Observed Result

Predicted Result????

Enco

ding

(Sem

antic

s)

(Pragmatics)

“The most direct and in a sense the most important problem which our conscious knowledge of nature should enable us to solve is the anticipation of future events, so that we may arrange our present affairs in accordance with such anticipation”. (Hertz, 1894)

Hertzian modeling paradigm

See: Modeling the World Lecture Notes


INDIANAUNIVERSITY

I501introduction

to informatics


The Antikythera Mechanism 2,000-year-old astronomical calculator

bronze mechanical analog computer discovered more than 100 years ago in a Roman shipwreck, was used by

ancient Greeks to display astronomical cycles. built around the end of the second century BC to calculate

astronomical positions With imaging and high-resolution X-ray tomography to study how it

worked. complicated arrangement of at least 30 precision, hand-cut bronze gears

housed inside a wooden case covered in inscriptions. technically more complex than any known device for at least a millennium

afterwards.

Not a universal Turing machine, but an analog computer


INDIANAUNIVERSITY

I501introduction

to informatics


let’s observe nature!

What do you see? Plants typically branch out How can we model that?

Observe the distinct parts Color them Assign symbols

Build Model Initial State: b b -> a a -> b a -> ab

Doesn’t quite Work!Psilophyta/Psilotum

bab

bb

bb

bb b

aa

aa

a aa


INDIANAUNIVERSITY

I501introduction

to informatics


Fibonacci Numbers!

Our First Model Rewriting production rules

Initial State: b b -> a a -> ab

n=0 : b n=1 : a n=2 : ab n=3 : aba n=4 : abaab n=5 : abaababa n=6 : abaababaabaab n=7 : abaababaabaababaababa

The length of the string is the Fibonacci Sequence 1 1 2 3 5 8 13 21 34 55 89 ...

Fibonacci numbers in Nature http://life-inspired.blogspot.pt/2015/02/links-for-modeling-

principles-of.html Romanesco: http://www.wussu.com/fractals/romanesco.htm


INDIANAUNIVERSITY

I501introduction

to informatics


http://pithemovie.com

Mathematics Is The

Language Of Nature


INDIANAUNIVERSITY

I501introduction

to informatics


what about our plant? An Accurate Model

Requires Varying angles Varying stem

lengths randomness

The Fibonacci Model is similar Initial State: b b -> a a -> ab

sneezewort

Psilophyta/Psilotum

bab

bb

bb

bb b

aa

aa

a aa


INDIANAUNIVERSITY

I501introduction

to informatics


Assignment I

Due October 11th

Focus on uncovering quadrants using data

collection and induction.

Propose a formal model or algorithm of what each quadrant is doing. Analyze, using

deduction, the behavior of this algorithm.

Q1 Q2

Q3 Q4

The Black Box: Due October 11th, 2017


INDIANAUNIVERSITY

I501introduction

to informatics


next class

Week 6 Lecture

Klir, G.J. [2001]. Facets of Systems Science. Springer. Chapters: 1,2,3. Optional: Chapters 8, 11

Lecture Notes Formalizing and Modeling the World

Presentations & Discussion Gauvrit et al (2017). “Human behavioral complexity

peaks at age 25”. PLOS Computational Biology, 13(4), e1005408. Kresnye, Cassie

Schmälzle et al (2017). “Brain connectivity dynamics during social interaction reflect social network structure”. PNAS, 114(20), 5153–5158. Wong, Vincent

readings (available in Canvas and online)

Info introduction to [email protected] informatics.indiana.edu/rocha/academics/i501/...

Documents

Transcript of Info introduction to [email protected] informatics.indiana.edu/rocha/academics/i501/...