Self-Programming: Operationalizing Autonomy Eric Nivel, Kristinn R. Thorisson Reykjavik University...

Self-Programming:Operationalizing Autonomy

Eric Nivel, Kristinn R. Thorisson

Reykjavik University

{eric,thorisson}@ru.is

AGI '09

I. Structural Autonomy = Self-Programming

II. Ikon Flux: a Proto-Architecture forSelf-Programming Systems

III. Conclusion & Future Work


The ChallengeTo build systems that

> Operate in real-world environments Rich, unstructured, real-time, partially specified, significantly changing.

> Adapt to change with limited human intervention Adaptation = maintain & improve the utility function: redefine their own mission and the means to fulfill it. Self-modification.

> Learn to modify and extend themselves Engineers cannot code everything beforehand: complexity barrier. Self-construction.

> Are aware of (reason about) what’s going on in the world & inside Reason with incomplete knowledge & limited resources1. Self-awareness.

> Encompass a broad scope A lot of sensors/actuators, skills, behaviors, etc. A lot of tasks & responsibilities, in a wide variety of situations and constraints. Scalability and integration matter.

We propose a generative approach to Architecture: evolve the structure to adapt behaviors

1. Wang P. (2007) The Logic of Intelligence. In Artificial General Intelligence. Springer Verlag.


Theory

> Operational closure (Varela & Maturana1). The structure (an assembly of components) creates a network of processes that modify the structure (by producing components and impacting the metabolism). Metabolism occurs in a topological domain (space + laws for production and modification). Dissipative systems: organization is maintained despite flows of energy & matter. Interaction with the environment raises the need for modification of the behaviors hence of the structure.

> Semantic closure (Pattee3, Rocha4) The organization (structures, processes, etc) is interpreted as instructions to perform the next evolutionary step. Semantic realization, e.g. self-preservation. ~ “Intentionality”.

1. Maturana H. R. and Varela F. J. (1980) Autopoiesis and Cognition: The Realization of the Living. Boston, MA: Reidel2. Eigen M. and P. Schuster P. (1979) The Hypercycle: A principle of natural self-organization. Springer, Berlin3. Pattee, H. H. (1995) Evolving Self-Reference: Matter, Symbols, and Semantic Closure Communication and Cognition. Artificial Intelligence, 12(1-2).4. Rocha, L. M. (2000). Syntactic autonomy, cellular automata, and RNA editing: or why self-organization needs symbols to evolve and how it might evolve them. In Closure: Emergent Organizations and Their Dynamics. Chandler J.L.R. and G, Van de Vijver eds. Annals of the New York Academy of Sciences, 901:207-223.

Eigen’s Hypercycle2


Self-Modification

> An informed evolutionary process of observation and control Planned to achieve goals relevant to the utility function. To realize the function and to remain able to do so. A continuous situation assessment.

> Fueled by the complexity of the world (through interaction) Passive mode, e.g. observing behaviors + performing analogy between the observed entity and the self. Proactive mode, e.g. other systems communicating with the self (direct orders, direct transfer of raw knowldege).

> Constrained by resource usage Resources are always scarce, choices have to be made. Handling problems at hand vs investing CPU for the future.

> Targeted at goal achievement Meta-goal defined by the designer. Survival is a sub-goal.

> Not natural evolution

individuals in a populationrandomness, genetic materialmutation, selection, cross-over

survival ?millions of years

?

TargetFuel

MeansTeleology

Time spanSpark of

life

the organization of an individual

interaction in the worldlearning, planning,

simulationfulfilling the missionmission time-horizon

bootstrap code

Natural Systems Engineered Systems


In Practice

> Mapping autopoiesis to a computational substrate Everything is programming: learning = modifying existing code or producing better one; planning = producing a plan (program) and a program to execute it; generating a mission = producing goals; etc. Component production code synthesis.

> We need additional classes of objects To support observation & control: models, processes & goals. To support abstraction: functions, markers. A reasonable granularity WRT the complexity of the synthesis rules and semantics.

> Bootstrap code Initial ontology, models, internal drives, meta-goals, programming skills, exemplary behaviors.

Component

Process

Network of processes

Energy & matter (fuel)

Toplogical domain

Semantic realization

Program

Execution of a program

Graph of causal dependencies between processes (generation)

Models, constraints observed in the world (incl. the self, and the self in the world).

Space: RAM holding code + I/O.Enables synthesis as an observable and controllable process. Traverses increasing levels of expression:

> low level: synthesis rules, syntax and semantics.

> higher levels: goal and plan generating programs and programs for measuring progress.

Achieving meta-goals (initially defined by the programmer)


Summary

> A Computational Autonomous System is a system

- Situated,

- Performing in real-time,

- Based on and driven by models,

- Operationally and semantically closed:

> The operational closure is a continuous program/model synthesis process.

> The semantic closure a continuous process of observation and control of the synthesis, that results from the synthesis process.

> Self-programming is the global process that implements both the operational & semantic closures From an engineering perspective:

a global and never-terminating process of architectural synthesis

II. Ikon Flux

Computational Substrate

> Platform abstraction Implict parallelism, memory allocation, garbage collection, distribution, synchronization, load-balancing.

> Machine-readable code Transparent code. Stateless, no side-effects. Explicit operational semantics. Low complexity (granularity) at the atomic level.

> Expressivity Functional language. Can represent equations (1st order); forward & backward chaining. Multiple LOD: pattern-matching. Abstraction: functions, processes, models, states, sub-systems, ... Dynamic types: ontology and its semantics allowed to change over time. Simulated execution is supported.

> Simplicity Operators are polymorphic. Easy to combine structures. Models are executable: no need for 2 // systems of representation. Ill-formed code fails gracefully.

mkr:sys ( ...)

mk.ins ( )

cat1: sys (...)

object: sys (... ...)

( )

II. Ikon Flux

Executive

> Data-driven Programs ~ production rules. Both program AND input data. Object reduction triggered by pattern-matching. Programs react to ANY object matching the pattern. Control values:

- activation: for programs,- intensity: for everything (potential inputs),- resilience: for everything.

Programs inject new code in the system. Growth counter-balanced by decay.

> Parallel Execution is step-locked; stack-based I/O polling. ~ Cellular automata, simulation runs. Finely grained. // search.

> Reflective The executive injects objects reflecting:

- synthesis operations, - function calls, - resource usage.

Processes captured as reduction graphs and vice versa. Processes are full-fledged objects. Causal relations between internal processes.

> Real-time

> Does not evolve

Time

Environment

External hardware /software sub-systems

Inputs

Outputs

Process

Outputs

Inputs

Executable code (program)

Self

II. Ikon Flux

Semantics

> State A stable regime of operation: a process. Of the self, of the world. From atoms (code existence) to complex phenomena: causal relations (explicit), temporal recurrence (implicit). Abstract: expressed as a pattern.

> State space A topological space to represent the contributions (+ or -) of objects to the achievement of an arbitrary reference process. Unit: time.

> Goal / constraint A set of regions in the state space. A process not implemented yet: internal drive, a.k.a. pregnance1,2. Abstract: expressed as a pattern.

> State space dimension IR U {reference process}. Contribution of O: distance (time) of O in a reduction graph, counted from the achievement of the reference process, e.g. pregnance satisfaction. Projection is either automatic (performed by the executive) or programatic (assertive, performed by the self).

> Saliency Computational (explicit): high/low contribution toward process achievement. Geometrical (implicit): idem for an observation process, e.g. finding regularities in a audio stream. Saliency detection has goal semantics.

1. Thom R. (1972). Structural Stability and Morphogenesis. Reading, MA: W. A. Benjamin.2. Thom R. (1990). Semiophysics: A Sketch. Redwood City: Addison-Wesley.

e

p5

-

[E]

t0

p4 t3

P

d,p3

c,p2

a,p0

b,p1

t4-t1

t4-t2

t4-t3

0

t4-t0 e,p5

at t’(now)

future

past

P at t

a

p0

p1

b

x

c

p2+

d

p3

p4-

PP

x=P(a,b)

[A]

t3

[B]

t4

[C] [D]

t1

t2

P

[A] [B]

[P(A,B)]

[A]

d,p3

c,p2

a,p0

b,p1

t3

a,p4

t4-t1

t4-t2

t4-t3

0

-(t4-t3)

at t

p0

p1

P

[A]

[B]

[p0(A)]

[p1(B)]

II. Ikon Flux

Semantics (cont.)

Positivecontributio

ns

Negativecontributio

ns

II. Ikon Flux

Semantics (cont.)

> Control Control values (int, act) computed for an object on a root-dimension as functions of its contributions to processes. Evaluated against thresholds (intensity & activation) binary activation. Define system configurations. Fast switching between configurations.

R1

01

sys-realm

0

1

R2

012

10

[

[

[

P1

P2

O1O2

O1

II. Ikon Flux

Semantics (cont.)

> Coupling A dimension is an object, projectable on another dimension (C(P)P’ = C(O)P x C(O)P’, O best for P). Mutually embedded spaces. To capture/express dependencies between processes.

a0

p1 b0

x0

P

[B]p4 -

[E]

P’’

c2

runtime t2

runtime t2,t1,t0

p0p2 p3[A]c1 [C]

d0

[D]

x1

P’

[A][C]

runtime t1,t0

[

b0,p1

a0,p0

P

P’P’’

d0,p3p2,p0

p2,p4c2,p2

P’

P’’

P’’P’

[

[

[

[

[

II. Ikon Flux

Semantics (cont.)

> Production of meaning Thom’s Semiophysics1: A process means another whenever it assumes the same pragmatics on the system. Dynamics: pregnance channelling. Case study: Pavlov’s experiment.

1. Thom R. (1990). Semiophysics: A Sketch. Redwood City: Addison-Wesley.

digest

[[

[[

chew

[[

salivate

chew, p2

p3

nutriments

[[

pre:hunger

salivate

p2

digest

p1, consume energy

p0

digest

meat within reach

visual attn.

[[

sln:visualattention

meat in sight

digest

salivate

consume energy

hunger: [intensity(energy)>T0]

p0

p1

energy

chew

nutriments

+

+p2

p3

[activation(digest)<T2]

[activation(salivate)<T3]

+-

[intensity(energy)>T0]

[intensity(energy)<T1]-

+

[food within reach]

visual attention [food at location L] [object at location L

object suitable for consumption]

a

a

ii

i

i Simplified model forthe pregnance hunger

p0

[x at t]

[xet p at t+dt]

[p: sln]

[x at t]

[prd xet p at t+dt]

F(x,p,dt)

[fmd]

p1

[F(x,p,dt1)]

[x at t]

[xet p at t+dt2]

[p] [F(x,p,avg F(x,p,dt:))]

-

p2 [F(x,p,dt)]

[r: rlm p]

[pji r x dt]

r

[[

sln:visualattention

meat in sight

[[

sln:auditiveattention

bell rings

audit. attn.

p3

p4

[p: preintensity(p)>T

r: rlm p]

[prj: pji r x: pror2: rlm x]

r2+

[r: rlm pintensity(r)>T]

[prj: pji r x: pror2: rlm x]

r2+

i

i

p5

[r: rlm pintensity(r)>T]

[prj: pji r x:]

r+

[x]

i

II. Ikon Flux

Semantics (cont.)

> Applications Behaviour conditioning. Modeling of causal relations. Semiotic memory. ...

Simplified model forpregnance chanelling

Builds a forward model Updates the model

Propagate intensities across dimensions (starting from the pregnance)

Updates the state space

learning

II. Ikon Flux

Organization Forms (exerimental !)

> Dynamic higher-order structures Identified / produced / demoted by code external to the target structure (stems form bootstrap). Morphogenesis (closures & stability) observed / controlled in the state space. Dynamic Architectural Synthesis.

S

O

Hierarchical

S0

S1

OS2

Flat

Structure construction

Functionstable coupling over

time of inputs and effects

II. Ikon Flux

Organization Forms (cont.)



Organonaggregate of code forming a substrate for a function

II. Ikon Flux





Organismaggregate of organons operationally closed for a (set of) function(s)

II. Ikon Flux





Individualaggregate of functions, organons and

organismssemantically closed for a (set of)

pregnance(s)

II. Ikon Flux


Organismaggregate of organons operationally closed for a (set of) function(s)

II. Ikon Flux

Sub-Systems

> Internal sub-systems

F0

F1

F0

F1

F0

F1

II. Ikon Flux

Sub-Systems (cont.)

> Internal sub-systems Recursive organization forms. Realize activities of either the self or entities in the world. Top-down (development model in the bootstrap). Bottom-up (emergent, identified/controlled). Can be allocated dynamically an instance of the executive. Can be kept away from evolution.

hand-coded

Action / reaction loop

faster, reactive

- concrete

slower, modeling or planning

- meta

Internal sub-system

Environment

External sub-

systems

II. Ikon Flux

Sub-Systems (cont.)

> External sub-systems Aggregates of axiomatic functions (exogenous code). Side-effects. Run on separate machines in their native implementation. Wrapped in a dedicated translator. The self can attempt to model external sub-systems. Programmers can help by injecting causality (command outcome).

II. Ikon Flux

Implementation

> Field-testing Full scale: breadth first and progressive increase in details and complexity. Loki: a crew member in a theatrical production. Responsible for the whole stage control.

- in a rich, noisy, unstructured environment under significant change,- to control a complex machinery,- interacting with people,- during long time periods (20 days),- incremental development (learning, integration),- supervised (rehearsals) and unsupervised mode (live performances).

No formal evaluation: time / $ pressures => no evaluation procedures / benchmarks.

> Current limitations Still a large amount of manual labor. No general determinsitic methods to identify & control the formation of organization forms (ad-hoc code).

III. Conclusion & Future Work

> An experimental framework to build strong autonomous systems Field-tested in real-world conditions. Theory is realistic. Non-trivial systems can be built. Full autonomy is still far away – but we’re really getting closer.

> Methodological issue: formation of higher-order structures in general deterministic ways Investigate modeling self-programing using Goldfarb’s Evolutionary Transformation System1. Ultimate goal: to develop an actual methodology for engineering arbitrary systems.

> Build more full-scale real-world systems Control of complex industrial systems, software intensive systems, developmental robotics, ...

1. Goldfarb L., Gay D. (2005) What is a structural representation? 5th variation, Faculty of Computer Science, University of New Brunswick, Tech. Report TR05-175

sys ( ...)

pgm1: pgm ( )

( )

(nil )inj (core )

mk.ins ( )

sys ( ...)

mk.ins ( )

cat1: sys (... ...)

cat2: sys (...)object: sys (... ...)

mk1: ( )

sys ( ...)

mk.ins ( )

mk2: ( )

pgm1: pgm // transitivity of instance relations spc (sys :object :mk1 ::) (red mk1 pgm spc in1: (sys (mk.ins object :a) :mk2 ::) (red mk2 pgm spc in2: (sys (mk.ins a :b) :int :act :res :) nil ((nil in2)) ) ((nil in1)) ) ((nil inj (core (sys (mk.ins object b) act int res))) )

II. Ikon Flux

Execution Model (cont.)

Deep pattern-matching

II. Ikon Flux

Execution Model (cont.)

( )

pro (...)

p1: sys ( ...)

sys ( ... ijt1)

( )

mk.xet ( ...)

pro (...)

p2: sys ( ...)

sys ( ... ijt2)

( )

mk.xet ( ...)

(|= )(= )

ifn ( )

( )

System-wide pattern-matching

ifn // detection of 2 arbitrary processes terminating at the same time spc nil ((|= p1 p2) (= ijt1 ijt2)) () spc p1: sys ((pro ::) mk1: ::) (red mk1 pgm spc (sys (mk.xet p1 ::) : : :ijt1) nil ((nil p1)) ) spc p2: sys ((pro ::) mk2: ::) (red mk2 pgm spc (sys (mk.xet p2 ::) : : :ijt2) nil ((nil p2)) )

Self-Programming: Operationalizing Autonomy Eric Nivel, Kristinn R. Thorisson Reykjavik University...

Documents

Transcript of Self-Programming: Operationalizing Autonomy Eric Nivel, Kristinn R. Thorisson Reykjavik University...