Joep van Liempd Communication & Multimedia Design, Creative Technology Academy for Media & User Experience, Avans Hogeschool, Breda March 2009

2

// CONTENTS 

CONTENTS ............................................................................................................................................... 2

INTRODUCTION ....................................................................................................................................... 3

EMERGENCE ............................................................................................................................................ 4

HISTORY .............................................................................................................................................................. 4

INTERDISCIPLINARITY ......................................................................................................................................... 7

CITIES .................................................................................................................................................................. 8

NEXT ................................................................................................................................................................. 10

MODERN MEDIA ................................................................................................................................... 11

BOIDS ................................................................................................................................................................ 11

MASSIVE ........................................................................................................................................................... 13

GAMES .............................................................................................................................................................. 17

SOCIAL NETWORKS & SCIENCE ............................................................................................................. 19

WORLD WIDE WEB ........................................................................................................................................... 19

EVOLUTIONARY ALGORITHMS ......................................................................................................................... 20

CONCLUSION ......................................................................................................................................... 22

REFLECTION ........................................................................................................................................... 24

REFERENCES & BIBLIOGRAPHY ............................................................................................................. 25

3

// INTRODUCTION  

A few years ago, in an introductory Human Sciences class, I read part of a book called Out of Control: The New Biology of Machines, Social Systems, and the Economic World1, written by Wired executive editor Kevin Kelly. The part I read explained, in general, a phenomenon the writer called hive mind. In short, he used the clear analogy of a bee hive and the intriguing behaviour a swarm of bees exhibits, especially (but not exclusively) when swarming outside its hive, to illustrate how something highly complex can emerge from a bunch of relatively dumb components. There’s no separate force overseeing or governing the swarm’s movements and actions, but clearly it does have its own behaviour, rising above the individuals it is composed of. This is what we call emergent behaviour: behaviour emerging from the crowd, coming into existence not directly out of the sum of its parts, but arising from a system of agents following relatively simple rules.

I had always found the wave-like movements of such airborne swarms an interesting sight, but after having read the aforementioned text I found myself increasingly attracted to phenomena of this nature. While reading up on the subject matter it didn’t take me long to realize that systems that exhibit emergent  properties are found all around us, not only in decidedly natural, highly visible circumstances such as the bee or ant colony, the flock of birds or the school of fish. You can find these self‐organizing  systems everywhere if you know where – and how – to look. Brains, cities, online social networks...these are all systems that, in one way or another, are made up of interconnected parts that, individually, behave in no way like the entity they make up does.

This concept as a whole is not new by any means (in fact the term emergent was coined in 1887 by a philosopher/psychologist named G. H. Lewes2), but it has been only in very recent times that our understanding of it has grown to a point at which we are beginning to harness and apply it ourselves.

Today, self-organizing systems are, for the most part, researched in academic and industrial contexts. However, recent developments have yielded practical uses in digital media as well. To be able to recognize and understand these applications, understanding what the phenomenon called emergence really comprises is essential. What is emergence really, and how can it be used to our benefit in the field of digital media?

1 Kelly, Kevin (1994), Out of Control: The New Biology of Machines, Social Systems, and the Economic World, Perseus Books, ISBN 0-201-48340-8 2 Lewes, G. H. (1875), Problems of Life and Mind (First Series), vol. 2, Trübner

4

// EMERGENCE  

  

Though what is described above might seem mysterious at first, the above (a passage from New York-based science/technology/society writer Steven Johnson’s introduction to his book Emergence3) actually describes a perfectly natural occurrence. It turns out this slime  mold (Dictyostelium  discoideum) is actually made up of a great number of individual single-celled organisms that normally live out their lives separately. Whenever their environment dries and warms up to a point at which they can’t gather enough food on their own, they coalesce into a single, larger creature that can last through the scarcity. Following that, when the environmental conditions return to their more hospitable state of damp coolness, ‘it’ returns to a state of ‘they’, until the need for coalescence arises again.

/ HISTORY In 2000, Japanese scientist Toshiyuki Nakagaki and two colleagues set up a maze with some food at the entrance and exit. Then, they placed a slime mold specimen (of a different but similar species to the aforementioned Dictyostelium, named Physarum  polycephalum) in the centre. Within hours, the slime mold had shaped itself in a line following the shortest route from the one food source to the other, maximizing its foraging efficiency. In doing so it had effectively navigated the maze and thereby solved the puzzle, without the help of a centralized brain to use for navigation. Though this particular event received quite some press coverage, it was only the latest in a line of experiments regarding slime mold behaviour and emergence in biology (and many more have been done since).45

3 Johnson, Steven (2002), Emergence: The Connected Lives of Ants, Brains, Cities, and Software, Scribner Book Company, ISBN 0-684-86876-8, p. 11-23; book referred to hereafter as ‘Johnson, Steven (2002), Emergence’ 4 Nakagaki, T. et al, Maze‐solving by an amoeboid organism (from Nature, 407: 470 (2000))

“Walk  through a normally cool, damp section of a  forest on a dry and sunny 

day, or  sift  through  the bark mulch  that  lies on a garden  floor, and  you may  find a 

grotesque  substance  coating  a  few  inches  of  rotting wood. On  first  inspection,  the 

reddish  orange  mass  suggests  that  the  neighbor’s  dog  has  eaten  something 

disagreeable, but  if you observe  the  slime mold over  several days – or, even better, 

capture it with time‐lapse photography – you’ll discover that it moves, ever so slowly, 

across  the soil.  If  the weather conditions grow wetter and cooler, you may  return  to 

the same spot and find the creature has disappeared altogether. Has  it wandered off 

to some other part of the forest? Or somehow vanished  into thin air,  like a puddle of 

water evaporating?” 

Nakagaki’s slime mold experiment; the big yellow blobs marked AG are food sources, the yellow lines through the maze are the slime mold. (Photo: Nature, September 2000) 

5

Slime molds had attracted scientific attention from multiple disciplines for a long time before then, mostly for being an intriguing example of coordinated group behaviour. However, until the late 1960s it was unknown how exactly that group behaviour worked. In 1968, physics Ph.D. Evelyn Fox Keller was trying to find ways of applying mathematics to biology at the Sloan‐Kettering  Cancer Center in Manhattan. While working there she met Lee Segel, an applied mathematician who shared her interests, and they started combining their knowledge and expertise regarding their respective fields of study. Together, they looked at the slime mold problem as a case study for what they were trying to achieve.

Up till then, the consensus among the scientific community had been more or less that the gathering process of slime mold cells was triggered by a number of so-called pacemaker  cells that would ‘order’ the aggregation to their peers at a given time. Research showed slime mold cells emit a substance called cyclic  AMP (or cAMP) that somehow tied into their behaviour, and it was hypothesised that these pacemakers used that substance to set off the gathering behaviour. This was a very logical assumption, parallel to much of the world around us that seems to work with a similar hierarchical system: our bodies seem largely governed by our brains, companies have managers and social organizations in general have ‘pacemakers’ in the form of kings, mayors, popes and other hierarchically important figures.

There was one problem though: no one could find the pacemakers in the slime mold. All cells were equal; there was no hierarchy to be found. For a long time this problem was thought to be attributable to poor research methods – but Keller and Segel were smart enough to ‘think outside the box’. Well over a decade before they had met and started working together, renowned English mathematician Alan Turing had written a paper on something called morphogenesis, which is the ability of organisms to form complex structures from simple beginnings (such as the embryonic development of an organism)6. In summary, Turing used mathematical tools to describe how complex organisms could assemble themselves without any central planning. So, what if the leading theory was wrong and the cells were organizing themselves, without the use of anything centralized like pacemakers – similar to some of the workings described in Alan Turing’s morphogenesis paper?

That stroke of inspiration turned out to be spot-on. When they drew up (on paper; at the time, there were no advanced digital simulation/visualisation tools like those we have now) some of Turing’s equations, they found that if each individual slime cell altered its cAMP output according to its individual environmental conditions and followed any trails of the pheromone it encountered, group behaviour would emerge. If the cells put out enough cAMP, clusters of cells would start to form. Cells would follow trails created by other cells and secrete their own trails concurrently, thus creating a feedback loop, creating ever larger clusters. If each single cell would simply release cAMP based on its assessment of its immediate environment, changes in the global environment would likely result in the observed behaviour of aggregation and subsequent dispersion. There seemed to be no need for special cells that decided the fate of the entire group after all.

5 Santiago Schnell, Ramon Grima, Philip Maini (2007), Multiscale Modeling in Biology (from American Scientist, March‐April 2007, Volume 95, Number 2) 6 A.M. Turing (1952), The Chemical Basis of Morphogenesis, Philosophical Transactions of The Royal Society of London, series B, volume 237

6

These days, the behaviour of the slime mold (pictured on the left, in aggregated state) is considered a classic case study in bottom-up behaviour. MIT professor Mitchel Resnick used the aggregation behaviour of Dictyostelium discoideum as a basis for what later became the StarLogo complex systems simulation package (pictured on the right, advancing stages of aggregation from top to bottom), using it to allow his students to experience the effects of altering agents’ pheromone output thus introducing them to emergent behaviour.9

For applied mathematicians and people familiar with fluid dynamics, this concept was easy to grasp and accept, because they were used to thinking in terms of bottom-up organisation. When trying to explain it to biologists, however, it was not so easy to convince them: they stubbornly kept on asking where the pacemaker was. They weren’t just used to thinking in hierarchical, top-down structures; it was completely embedded in their thought process. It was not until roughly 10 years later, at the end of the 1970s, that sufficient experimentation had proven indisputably that the cells were essentially organizing themselves, from the bottom up instead of from the top down.789

7 Fox Keller, Evelyn (1996), The Force of the Pacemaker Concept in Theories of Aggregation in Cellular Slime Mold In Reflections on Gender and Science, New Haven, Conn.: Yale University Press 8 Holland, J. H. (1999), Emergence: From chaos to order (helix books), Perseus Books Group 9 Resnick, Mitchell (1999), Turtles, Termites, and Traffic Jams: Explorations in Massively Parallel Microworlds, Cambridge, Mass., and London: MIT Press

7

/ INTERDISCIPLINARITY Keller and Siegel’s research eventually lead to huge developments and other people studying the effects of emergence in various disciplines and linking those disciplines together. In 1984, an institute dedicated to the study of complex systems named the Santa Fe  Institute (after its city of residence in New Mexico, United States) was founded. Its goal was set to disseminating the notion of a separate interdisciplinary research area called Complexity  Theory (referred to at SFI as Complexity Science).

Some of the relevant fields were already being studied much earlier on though. Scholars like Adam Smith (18th century economist), Friedrich Engels (19th century philosopher/social scientist), Charles Darwin (19th century naturalist) and the earlier mentioned Alan Turing (early 20th century mathematician) were already producing writings that revealed emergent behaviour in the systems they analyzed. However, back then, these weren’t recognized as having some essential points in common and as a result the works were never actively associated with each other before the 1970s.

So what exactly do the systems these men were researching and hypothesizing about have in common? When we look at the bigger picture and put them side by side, we find they each overcome problems, adapt and continue their existence by depending on the connected behaviour of large amounts of separated elements that are not aware of the system they are part of, instead of using a single intelligent source for government and direction.

Steven Johnson’s book Emergence (which I referenced earlier in the introductory chapter) provides a very interesting view regarding self-organizing systems. In it, Johnson brings together a range of subjects that are not commonly associated with each other, with the goal of explaining and clarifying the phenomenon of self-organization. Some of these subjects cover natural and physical occurrences, ranging from Dictyostelium  discoideum  to ant colony behaviour; others tend to instances found in today’s culture. As Johnson states, these systems are “complex adaptive systems that display emergent behaviour”10. This means that in such a system, agents operating on one scale instigate behaviour that lies on a scale above them. Smith touched upon this when he wrote about the ‘Invisible Hand’ of self-regulation within economic systems. A metaphorical example he gives is that of the Butcher, the Baker, and the Brewer providing goods and services to each other out of self-interest; the unplanned result of this division of labour is a better standard of living for all three11. Darwin is of course famous for his theory of evolution, which also displays emergence: the evolution of a species emerges from mutations that occur over generation upon generation in the individuals that make up that species. The species as an entity adapts to its (changing) environment because the parts it is made up of change over time.12

10 Johnson, Steven (2002), Emergence, p.17 11 Smith, Adam (1776), An Inquiry into the Nature and Causes of the Wealth of Nations, London: Methuen and Co., Ltd., ed. Edwin Cannan, 1904. Fifth edition 12 Kauffman, S. A. (1993), The origins of order: Self‐organization and selection in evolution, Oxford University Press

8

/ CITIES With Engels, it’s the notion that cities’ neighbourhoods are entities that exhibit behaviours like growth, contraction, movement and division without their inhabitants controlling these directly and which are beneficial to those inhabitants. Engels’s case is perhaps a bit abstract for the untrained viewer, but it is an important one because through its abstractness it illustrates how abstract yet at the same time glaringly obvious an emergent system can be. Cities, their nature, and that of the individual parts they are made up of, have been studied ever since they started forming. However it was only during and after the industrial revolution (late 18th- early 19th century), when multiple cities grew explosively – starting in Britain with the effect subsequently spreading all over Europe – that some studies began to take note of effects they could not explain directly through then-current knowledge. Quoting from Engels’s (co-founder of communist theory, alongside Karl Marx) ‘The Condition of the Working Class in England in 1844’:

 

13

Next to displaying the difficulties of thinking in models of self-organization, this passage shows that the city exhibits emergent properties. Engels went to Manchester to work for the family business (a textile firm) there during the industrial revolution and was surprised by how well it hid its dark side (the atrocious conditions of the working class) from upper class eyes, yet it “has been built less according to a plan” than other cities. This seemed like a contradiction at the time, hence his – probably well-justified – expression of mistrust towards the bigwigs. However, as he already stated himself, the city had been built too quickly, with too little space for ‘trivialities’ such as neighbourhood planning. As Engels correctly observed, the system of separation on display in

13 Engels, Friedrich (1844), The Condition of the Working Class in England in 1844, Oxford University Press (1999), ISBN 0-192-83688-9

“The town itself is peculiarly built, so that someone can live in it for years and 

travel into it and out of it daily without ever coming into contact with a working‐class 

quarter or even with workers – so  long, that  is to say, as one confines himself to his 

business  affairs  or  to  strolling  about  for  pleasure.  This  comes  about mainly  in  the 

circumstances  that  through  an  unconscious,  tacit  agreement  as much  as  through 

conscious,  explicit  intention,  the working‐class districts are most  sharply  separated 

from the parts of the city reserved for the middle class... 

I  know perfectly well  that  this deceitful manner of building  is more or  less 

common to all big cities.  I know as well that shopkeepers must  in the nature of the 

business  take premises on the main thoroughfares.  I know  in such streets there are 

more  good  houses  than  bad  ones,  and  that  the  value  of  land  is  higher  in  their 

immediate vicinity  than  in neighbourhoods  that  lie at a distance  from  them. But at 

the  same  time  I  have never  come across  so  systematic a  seclusion of  the working 

class  from  the  main  streets  as  in  Manchester.  I  have  never  elsewhere  seen  a 

concealment  of  such  fine  sensibility  of  everything  that might  offend  the  eyes  and 

nerves of  the middle classes. And yet  it  is precisely Manchester  that has been built 

less according  to a plan and  less within  the  limitations of official  regulations – and 

indeed more  through accident –  than any other  town. Still  ...  I  cannot help  feeling 

that  the  liberal  industrialists,  the  Manchester  “bigwigs,”  are  not  so  altogether 

innocent of this bashful style of building.”

9

Manchester had been formed solely by individual actions: the effect emerged from the actions of interconnected individuals (for example, wealthier individuals bought land and built houses near their upper class peers, and thus the value of the land there increased and the lower class was forced to live in less favourable parts), the streets’ layout had not been planned in advance. In modern society, as cities have become more abundant, this effect has only spread. Some cities were actually planned in great detail before they were built (Washington D.C., for example) but often we can find that neighbourhoods have grown, making cities ‘run’ more efficiently. This might not always directly affect the people who populate it, walking on its sidewalks, interacting vigorously, but the city itself, the system, benefits from it: growing, changing and adapting in order to continue existing.14

This movement from low-level rules and behaviour to higher-level sophistication is what is called emergence. Another word that is often used in related literature is complexity, which is a distinct term referring to the complex behaviour of these systems themselves, rather than to the way it is experienced by their individual parts (who may find the observation of such behaviour an overwhelming, complex experience).

14 Jacobs, Jane (1961), The Death and Life of Great American Cities, New York: Vintage

A street plan of a part of Manchester, made by Friedrich Engels. The structure shows how the big roads (Long Millgate, Fennel Street and Todd Street) are visually shielded from the slums in the centre of the block – which happened without having been planned beforehand.

10

/ NEXT Now that we’ve got some idea of what emergence is and what it can do, we can start looking at how it’s being used today and what the future might bring. Johnson reckons there are three phases in the development of our understanding of self-organization. Phase one represents the recognition of the phenomenon’s existence, dispersed over several different disciplines. Phase two consists of figures from multiple scientific disciplines seeing their peers from other disciplines looking at similar problems in self-organizing systems, and realizing the ‘problem’ transcended the traditional borders: “Self-organization became an object of study in its own right, leading to the creation of celebrated research centres such as the Santa Fe Institute, which devoted itself to the study of complexity in all its diverse forms.”15 Phase three is the one we started entering in the second half of the 1980s, when we started creating our own emergence, rather than just analyzing it. We began using artificial emergence in our software applications, video games, art... consciously putting together systems that make use of the laws of emergence.

 

15 Johnson, Steven (2002), Emergence, p. 21

11

// MODERN MEDIA

With the ever advancing computer technology came the interest of the entertainment industry. After first having toyed with some CG animation in films such as TRON and The Abyss, it was now starting to feel the need for more complex behaviour from their artificial cast members. A point came where it was impossible to still use hand-animation for every single digital element.

/ BOIDS In 1986, Craig  Reynolds developed the Boids algorithm, which allows for simulation of flocking behaviour of groups of digital agents in their digital environment. The way this algorithm works is surprisingly simple. Each so-called Boid constantly checks for the location of all its peers, and then looks at the exact position of any peers that are too close and moves away from them. Secondly it considers the speed of all other Boids, averages it and adjusts its own speed towards the resulting number. Lastly, it takes the positions of all agents within the environment, averages that (the result being the centre point of the flock) and moves towards the position resulting from that calculation.

With the right adjustments, the resulting behaviour looks remarkably similar to that of a flock of birds or a school of fish – and best of all, it’s highly adaptable. If you want to have the Boids steer along a path while they are flocking, simply create a set of points along that path and change the ‘centre’ position to that point in each reference frame (advancing one point each frame). That way, the flock will follow the path but its general movements will still look natural. You can then, for example, create a set of coordinates (which can be animated) that the Boids should avoid, and just

The Boids algorithm: these three simple rules create flocking behaviour.

12

add that to the position-calculation algorithm by telling the Boid to move away from those coordinates should they come within a certain range of them. When Reynolds was on the visuals crew for Tim Burton’s Batman Returns (1992), he used Boids to great cinematic effect for the CG bats and penguins that appeared in the movie. With the algorithm, he was able to make their animated movements appear autonomous, steering the flock in a certain direction without having to resort to any less efficient methods (like manually animating every single individual separately). This is a very visual example of making effective use of artificial emergence, creating a distributed, complex system capable of being unpredictable and lifelike.

While the wonders of digital animation and computer generated imagery were being discovered, studios soon found that it was often more effective to use autonomous systems for the animation of complex systems like crowds of people, instead of animating everything by hand.

Boids in Batman Returns: the movements of the bats (left) and the penguins (right) were all controlled by the rules of the flocking algorithm, making for natural-looking behaviour in large groups of computer-rendered characters – a first for the big screen.

13

One of the iconic battle scenes in the Lord of the Rings trilogy that make it clear where Massive got its name from.

/ MASSIVE Fast forward to 2001, when part one of Peter  Jackson’s film adaptation of the famous Lord of  the Rings book trilogy (by J.R.R. Tolkien) was released. The film (named The Fellowship of the Ring after its corresponding book) included several scenes depicting vast armies consisting of thousands of units going to battle. Particularly the film’s opening sequence (where the audience gets to see the armies of main antagonist Sauron fighting those of the Last Alliance of Men and Elves) required a new approach to animating such a large amount of individual agents to keep things looking real (or at least as real as an epic fantasy battle can look). In order to do this, Stephen Regelous (a computer graphics software engineer then working for Weta Digital, the company that provided the digital special effects for the trilogy) developed a system he called Massive (for Multiple Agent Simulation 

System in Virtual Environment16).

The first versions of Massive worked by taking a set of pre-programmed behaviours and animations Weta designed and assigning these to the individuals on the virtual battlefield, using an adaptable rule-set designed to create fighting behaviour for each soldier. Part of the rule-set for these battles was somewhere along the lines of “keep running until you run into an enemy unit, then engage it”; simple, yet effective. Because of the heavy use of approximation and fuzzy  logic in its calculations, the system allowed every single agent to assess the current situation individually and respond to it following its own set of rules without taking up ridiculous amounts of processing power. To get the desired effects of realistic crowd movement during the battle scenes, all the team needed to do was set up parameters like goals and the amount of characters in each group. The key here, again, is emergence. Massive was designed explicitly to make use of emergent behaviour, relying on lack of overall control and decentralized governance to achieve behavioural effects that closely resemble those that occur in real-life crowds.17

16 http://massivesoftware.com (2009), Massive Software 17 The Lord of the Rings: The Two Towers ‐ Extended Edition (2002), Part Four: The Battle for Middle‐Earth Begins ‐ Visual Effects ‐ WETA Digital Feature, Newline Cinema

14

After LotR  I went to theatres, the battle sequences gained widespread industry attention. It had never been done on such a scale with such realism before, and this new technique opened up a whole realm of possibilities. Following completion of the final part of the trilogy (The Return of the King), Regelous continued working on the program and set up a company called Massive Software, dedicated to the further development and sale of his creation. The version of Massive that is on the market now is the reason you’ve been able to enjoy such spectacular crowd- and swarm-related visuals in movies such as The Chronicles of Narnia: The Lion, the Witch and the Wardrobe (centaurs vs. minotaurs), 300 (half-naked men vs. monster ninjas) and The Dark Knight (bats). In addition, the past few years have yielded quite a few TV-commercials that make use of Massive’s qualities as a device for creating a feeling of ‘epicness’; notable examples include Carlton  Draught’s ‘Big Ad’ campaign and Axe’s (the deodorant, called Lynx in some markets) ‘Billions’ commercials (in which huge amounts of women dressed only in bikinis converge upon a guy on a beach who is spraying himself with Axe deodorant).

Carlton Draught’s ‘Big Ad’ (left) & Axe’s ‘Billions’ (right): absurd, comedic... it’s obvious the images couldn’t have been recorded with live actors, but the realistic crowd movement does make them look stunningly lifelike.

15

In addition to being able to create emergent systems tailored for the entertainment industry, the program now includes a complete interface that works with a tree-like structure for assigning behaviours, animations and other data to agents. This way, even designers who have no experience in programming can put together complex crowd animations. It has also been made compatible with other industry-leading hardware and software applications, partnering with parties like Autodesk (Maya 3D modelling/animation software), nVIDIA (dedicated graphics processing hardware) and Pixar Renderman (image rendering platform); support for the Python scripting language has recently been added as well.

It seems only natural that, because of the versatile nature of the program’s core qualities, the Massive project evolved to support more diverse applications. Massive Software recently officially expanded its active fields to include Architecture, Engineering & Construction (AEC) and Robotics, next to the original field of Media & Entertainment. Uses in AEC of course include the presentation of models (showing a fly-through of a building populated with active subjects obviously provides much more of an idea of what a structure will look and work like when ultimately finished), but are focused on the simulation of people, cars or anything that needs to occupy/move through a certain space. This way, behaviour in all kinds of situations can be predicted with reasonable accuracy and problematic structures (like a door turning out to be a bottle-neck at a critical point in a panic situation) can be adjusted easily before they are put to work in real life. Of course, this isn’t the first time by far that crowd simulation has been put to use to solve such problems, but Massive has a few abilities that provide a simulation that is closer to reality than other methods that are currently in use. By simulating senses of sight, hearing and touch for every agent, it provides what the company calls “more inherently natural behaviour”18. What they mean by this, is that by adding complexity of a specific kind to the simulated individuals and bringing them closer to their real-life counterparts, the behaviour of the entire group bears a greater resemblance to the behaviour of a group of actual people. This denotes a trend in the design of emergent systems: by learning from what works in naturally  occurring emergent systems and applying that gained knowledge by adapting it to the created system, applicability is improved without making many compromises.19

18 http://massivesoftware.com/architecture‐engineering‐construction (2009), Massive Software 19 http://www.massivesoftware.com/products (2009), Massive Software

“Massive Prime's intuitive node-based interface allows artists to interactively create AI-enabled agents with custom responses for specific behaviours, without any programming. $17,999 USD plus $3,999 per year 

for upgrades & support”19

16

Another good example of this is a recent Massive-related project, in which Hanson Robotics used parts of Massive’s programming to enhance their prototype robot’s behaviour:

20

21

Pushing boundaries like this has been going on in different ways in another area of modern media I’d like to address: that of video games.

20 Diane Holland (2008), CEO, Massive Software http://www.roboticstrends.com/home/features/robot_driven_software_brain 21 http://massivesoftware.com/robotics (2009), Massive Software

“..."Zeno," a robot that uses breakthrough artificial intelligence (AI) software to 

reason and get smarter over time. Hanson Robotics partnered with Massive Software, 

the  developer  of AI  simulation  software  used  in  films  such  as  the  Lord  of  the  Rings 

Trilogy, Happy Feet, Ratatouille, and more, to design a robot that is able to view a 3D 

mental  image  of  his  environment  to  determine  and  control  physical  action  and 

reactions. 

[...]"With  very  little  modifications  to  our  software,  they  are  controlling  the 

physical  actions  and  reactions  of  a  robot  in  the  real world, which  has  very  exciting 

implications for Massive in a variety of markets outside of the animation industry."”

Zeno’s face (prototype). The huge eyeballs facilitate two cameras that provide 3D-‘vision’.21

17

/ GAMES What we noticed happening in actual cities is also applicable to their virtual brethren. Almost parallel to the film industry adopting emergent systems for simulation, the video game industry started showing signs of realizing the potential of these systems as well.

Around 1984, Will Wright was designing levels for a game called Raid on Bungeling Bay. It was a simple game that was set up in a top-down view of a small planet; the player controlled a helicopter and had to bomb the antagonist’s bases, set on islands in an ocean, to save the planet. The islands were interconnected and contained behaviour that didn’t allow the player to just destroy the bases one by one (they could try, but that would give the other bases time to develop and gather enough weaponry to overwhelm the player when they would get to them). When Wright was working on the game, he noticed he found designing the islands much more enjoyable than actually playing the game. Being a programmer, he took his island-design toolset and turned it into a full city-building toolset, after which he developed an ecosystem that the city would reside and “live” in. The rules on which he based the ecosystem were in turn based on theories by Jay Forrester, who is known as the founder of System dynamics (which is “a method for studying the world around us. It deals with understanding how complex systems change over time. Internal feedback loops within the structure of the system influence the entire system behaviour.”22).23

That city-building toolset eventually became the famous SimCity. The rule set Wright developed, rooted in System dynamics, allowed for the system to evolve over time. If you’ve played SimCity before, you’ll know what I mean by that: once your city gets large enough, you’re really not directly in control of its behaviour. You may be able to steer it, nudge it in a direction. But the system, the city, responds to your actions on its own. Again, it does this without controlling the entire thing from one central place but by having all its parts interconnect and respond to each other. As such, a city in SimCity is always different even though the parts are the same every time you play. Yet again, the similarities to the previously-discussed are obvious – but so are the differences: this simulated emergent system operates in real-time, and it provides a more realistic simulation than any other known method could provide because the rules in its virtual world tap into the same laws of emergence that the real world adheres to.

22 MIT System Dynamics in Education Project, http://sysdyn.clexchange.org 23 GameSpot's SIMply Divine: The Story of Maxis Software, http://www.gamespot.com/features/maxis (2009), CBS Interactive Inc.

The original SimCity: at first sight, it might look like a simple simulation game; look closer and you’ll find that it has some powerful qualities that lie parallel to those of real-world emergent systems.

18

Long after creating the first instalment of SimCity, in February 2000, Maxis (Wright’s company) and Electronic Arts released the incredibly popular The Sims: a game that has players manage a family of characters in their day-to-day-life. The game became the best selling PC-game ever in March 2002 and hasn’t left that spot since. The Sims works like SimCity in a way, in that the game operates on a rule set to drive the game like a living system – but in this case, it actually needed to be limited in its independence from user input. During early testing, it became apparent that the game became uninteresting to players, because the characters were living their own life in the game’s environment just fine without input of the user. Wright: “One of our biggest problems here was that our AI was too smart. The characters chose whichever action would maximize their happiness at any given moment. The problem is that they’re usually much better at this than the player.” And so Wright had to dumb down his digital creations, making them focus on short-term gratification (like watching TV) instead of long-term goals (studying for a job promotion) and making their personality weigh in on their decisions to a very high degree (a neat Sim will spend way too much time cleaning up if left to themselves, a sloppy Sim will never clean up at all if the player doesn’t tell them to). This was enough to make the player a sorely needed component – ambition? balance? – of their world.24 This movement towards using emergence to our own ends and struggling with the balance between system control and system independence is consistent with the third phase proposed by Johnson.

   

24 Johnson, Steven (2002), Emergence, p. 188

19

// SOCIAL NETWORKS & SCIENCE

Our modern way of living in a big world made tiny through the beloved digital network of social interconnectedness we call the internet has yielded yet more areas of interest. Emergence: the web is full of it.

/ WORLD WIDE WEB If you’re looking for practical examples of emergence utilized in real-world applications, the internet’s various (social) networks are a good place to start. Take eBay, for example; “they had this problem of all these people conducting auctions with other people, and there were no ‘authorities’ in that mix. So [they] could either take a top-down approach and run a credit check on every single person who does business on [eBay] or [they] could let all the different agents rate each other and make those ratings visible to everybody else. This way, they have the community regulate itself rather than having some authority come in and watch over and survey the entire situation and arrest or promote people based on their observations.”25 They utilise the interaction (via their rating system) of every bidder and seller on eBay to create a system that, on its own, keeps frauds as well as honest people/stores easily spottable – negating the need for a separate authority to double-check every person and action (which would be near impossible).

News websites like Slashdot and Digg make use of a similar system to ‘filter’ their news by using ratings made by the public, and social bookmarking websites like del.icio.us and StumbleUpon recommend other websites based on the same thing. The striking thing about these is that they start out with a heap of unorganized, uncategorized information, and the disorganized actions of all participants (who are equal in importance and abide by the same rules) actually produce quality output in large amounts with minimal effort from the individual ‘agents’.26

From a more technical point of view (as opposed to the previously mentioned social-oriented), we can see systems like Amazon’s suggestion features and last.fm’s song recommendations. These recommender  systems get better as you use them more - they’re adaptive and ‘learn’ from the interaction with their users, both on an individual level and as a whole system. The more you use them, the more effective and accurate your recommendations get. The more individual users are part of the systems, the more effective the systems get, as a whole. To do this, they employ techniques and algorithms that are made to tap into the potential of emergent systems.27

25 The Roots of The Matrix – The Hard Problem: The Science Behind The Fiction (special DVD feature of ‘The Ultimate Matrix Collection), Warner Bros. (2003) 26 O’Reilly interview: Steven Johnson on “Emergence”, http://www.oreillynet.com/pub/a/network/2002/02/22/johnson.html (2002), O’Reilly Media 27 Toby Segaran (2007), Programming Collective Intelligence: Building Smart Web 2.0 Applications, O'Reilly Media, Inc., USA, ISBN 0-596-52932-5

20

/ EVOLUTIONARY ALGORITHMS These techniques and algorithms in turn connect us to the world of science, where find examples of a more abstract and technical nature that have deployed emergence to a degree for a longer time, outside  the realm of simulation. Evolutionary algorithms exhibit various emergent properties, the primary being adaptation. Being based on naturally-occurring genetics and evolution, the methods of computation and problem-solving used in evolutionary systems show characteristics along the same lines as their corresponding natural phenomena (some of which were discussed in the first chapter).

The way such systems are set up is partly based on their unpredictability: the very fact that it isn’t possible to predict what exactly their outcome (or outcomes) will be. They work their way toward a solution to a given ‘problem’ by adding and removing behaviours from the ‘gene pool’, mutating and recombining genes generation upon generation, thus using evolution to adapt their output to a set of conditions or goals. The hard part is determining how to tell which ‘strands of DNA’ are the best ones at any given time, which ones get to ‘reproduce’. In other words, determining what every strand’s fitness is, thus determining what properties will be in the next generation (since the fittest will survive, like they do in Darwinian evolution). Regardless of what the results are, though, that’s where the emergence manifests, like it does in nature’s brand of evolution: the final result from this process of ‘natural selection’ is a ‘species’ (a string)  that’s adapted (in comparison to what the system started out with) to the ‘environment’ (the conditions the process was set to meet or the goals it was set to try and accomplish) without any of the individual strings having a direct effect on the overall outcome.28

Computer graphics artist and researcher Karl  Sims used these algorithms in the creation of his Evolving Virtual Creatures, which are “results from a research project involving simulated Darwinian evolutions of virtual block creatures. A population of several hundred creatures is created within a supercomputer, and each creature is tested for their ability to perform a given task, such the ability to swim in a simulated water environment. Those that are most successful survive, and their virtual genes containing coded instructions for their growth, are copied, combined, and mutated to make offspring for a new population. The new creatures are again tested, and some may be improvements on their parents. As this cycle of variation and selection continues, creatures with more and more successful behaviours can emerge.”29

28 A.E. Eiben & J.E. Smith (2003), Introduction to Evolutionary Computing, Springer, ISBN 3-540-40184-9 29 K.Sims (1994), Evolving Virtual Creatures, Computer Graphics (Siggraph '94 Proceedings), July 1994, p. 15-22

Some of Sims’ Evolved Virtual Creatures, clockwise from the top left, their purposes were: following, 

swimming, 

competing for a 

green block, 

hopping 

21

Sims’ company GenArts, Inc. develops visual special effects software for the film and video industry. Their Sapphire plug-ins extend the capabilities of editing and effects workstations by providing digital artists with a collection of over 200 state-of-the-art image processing and synthesis effects, compatible with most industry standard software and hardware. These have been used in blockbuster special-effects movies like Iron Man, X‐Men: The Last Stand and Transformers, by big studios like Industrial Light & Magic and Sony Pictures Imageworks.30

30 GenArts, Inc. home page, http://www.genarts.com (2009), GenArts, Inc.

22

"The  internet  is  a  hugely  inefficient  way  to  do  anything.  Because  there  are 

tremendous  amounts  of  redundancy,  there  are  tremendous  amounts  of  'out‐of‐

control‐ness' because you have no idea where a message is going to go from A to B; 

it  can  go  out  the most  long‐winded way  possible. We  don't  care. What we  care 

about is that  it's a very robust system,  it's like a biological system in the sense that 

it's  very  hard  to  kill  or  to  harm.  The  price  for  that  is  that  things  happen  on  the 

internet that we can't really account for. And so, that's a system for which we have 

paid the price of letting it be slightly out of control in order to have it become better 

for us."25 

// CONCLUSION

Emergence is all around us, everywhere, every day. Your very own body might just be the epitome of it: all of your individual cells bound and binding together into this immensely complex system, distributed, no top-down central intelligence that constantly tells every part what to do or how to fit together with others. You, your central self, can make your finger move – but you don’t have to tell your finger not to fall apart. It just sticks together, it just lives, it just works, because all parts in the system do their thing, what they are programmed to do, and it works because they do no more than exactly that.

The artificial brand of emergence hasn’t reached such levels of excellence yet – as far as we know, anyway – but we are using it to our benefit, as I’ve shown, in a diversity of ways – though yet again not by far as many as can be observed in nature. And even though people do seem to find the uses, they do not always seem aware of the interdisciplinary connections that emergence brings forward. Now that you are aware of these things, you can take them into account when solving problems. Try thinking from the bottom up where you would normally have gone for the top down approach, don’t be afraid to let go of some direct control when a system has the potential to control itself.

When, during their 2008 Seed Salon talk, famed astrobiologist Jill Tarter asked Will ‘SimCity’ Wright whether his games contained real emergence, he replied: “well certainly, in fact, we rely heavily on emergence.” As I’ve shown, it’s becoming more and more of a factor in the (new) media industry, and the quick implementation and adaptation that is common to this field combined with the increasing importance of digital and interactive media bode for an interesting future regarding the subject.

These systems are robust, independent and adaptive, even self-sustaining – but they’re also unpredictable, hard to control and potentially hard to contain. To implement them effectively on a large scale will take research and development every time, but their potential efficiency and effectiveness could have profound effects on our way of life. Due to their inherently unpredictable nature, it’s hard to say what artificial emergent systems will bring for us in the future, but I hope I’ve been able to convey a sense of understanding about some of the things that make this world tick. I hope I’ve brought to your attention that using emergence as a tool is an opportunity that shouldn’t be underestimated – nor overestimated – and you might carefully consider taking that step of giving away some control to an artificial system, in exchange for adding some life to whatever it is you’re creating. Use with caution, and the effort might prove rewarding; like Kevin Kelly says in The Roots of The Matrix – The Hard Problem: The Science Behind 

The Fiction:

23

 

24

// REFLECTION

I have found learning about emergence a life-enriching experience. Emergence really is everywhere, and seeing it pop out like a stroke of nature’s brilliance every so often, seeing the way so many things are connected by this common feature, has changed and expanded my entire frame of reference. A lot of things were clarified; a lot of interesting questions were raised. I am not done with this subject yet, not by a long shot. In fact, I can say that the way I look at certain aspects of the world we live in has permanently changed – profoundly.

As I wrote earlier, I hope I’ve been able to convey at least part of that eye-opening experience to you, that I have whetted your appetite for it like that passage from Kevin Kelly’s book once did for me. If you’re looking for more material, I recommend you take a look at some of the material I referenced, particularly the documentary called The Roots of The Matrix – The Hard Problem: The 

Science Behind The Fiction (if you can get a hold of it), which touches upon many interesting topics regarding reality simulation, artificial intelligence and emergence (amongst others) and gives several interesting viewpoints from prominent figures from the academic, commercial and science fiction worlds. Another recommendation is Steven Johnson’s book Emergence. Regardless of how interesting this read has been for you, reading that book will get you thinking, at the very least.

Thank you for reading.

25

// REFERENCES & BIBLIOGRAPHY

Books 

Kelly, Kevin (1994), Out of Control: The New Biology of Machines, Social Systems, and the Economic World, Perseus Books, ISBN 0-201-48340-8

Lewes, G. H. (1875), Problems of Life and Mind (First Series), vol. 2, Trübner, ISBN 1-417-92555-8

Johnson, Steven (2002), Emergence: The Connected Lives of Ants, Brains, Cities, and Software, Scribner Book Company, ISBN 0-684-86876-8 

Holland, J. H. (1999), Emergence: From chaos to order (helix books), Perseus Books Group, ISBN 0-738-20142-1 Resnick, Mitchell (1999), Turtles, Termites, and Traffic Jams: Explorations in Massively Parallel Microworlds, Cambridge, Mass., and London: MIT Press, ISBN 0-262-68093-9

Smith, Adam (1776), An Inquiry into the Nature and Causes of the Wealth of Nations, London: Methuen and Co., Ltd., ed. Edwin Cannan, 1904. Fifth edition, ISBN 0-226-76374-9

Engels, Friedrich (1844), The Condition of the Working Class in England in 1844, Oxford University Press (1999), ISBN 0-192-83688-9

Jacobs, Jane (1961), The Death and Life of Great American Cities, New York: Vintage, ISBN 0-679-60047-7

Toby Segaran (2007), Programming Collective Intelligence: Building Smart Web 2.0 Applications, O'Reilly Media, Inc., USA, ISBN 0-596-52932-5

A.E. Eiben & J.E. Smith (2003), Introduction to Evolutionary Computing, Springer, ISBN 3-540-40184-9

Smaller publications 

Nakagaki, T. et al, Maze‐solving by an amoeboid organism (from Nature, 407: 470 (2000))

Santiago Schnell, Ramon Grima, Philip Maini (2007), Multiscale Modeling in Biology (from American Scientist, March‐April 2007, Volume 95, Number 2)

Fox Keller, Evelyn (1996), The Force of the Pacemaker Concept in Theories of Aggregation in Cellular Slime Mold In Reflections on Gender and Science, New Haven, Conn.: Yale University Press

K.Sims (1994), Evolving Virtual Creatures, Computer Graphics (Siggraph '94 Proceedings), July 1994, p. 15-22

A.M. Turing (1952), The Chemical Basis of Morphogenesis, Philosophical Transactions of The Royal Society of London, series B, volume 237

Kauffman, S. A. (1993), The origins of order: Self‐organization and selection in evolution, Oxford University Press

Web 

http://massivesoftware.com (2009), http://massivesoftware.com/architecture‐engineering‐construction (2009), http://www.massivesoftware.com/products (2009), http://massivesoftware.com/robotics (2009), Massive Software

Diane Holland (2008), CEO, Massive Software http://www.roboticstrends.com/home/features/robot_driven_software_brain

26

MIT System Dynamics in Education Project, http://sysdyn.clexchange.org

GameSpot's SIMply Divine: The Story of Maxis Software, http://www.gamespot.com/features/maxis (2009), CBS Interactive Inc.

O’Reilly interview: Steven Johnson on “Emergence”, http://www.oreillynet.com/pub/a/network/2002/02/22/johnson.html (2002), O’Reilly Media

GenArts, Inc. home page, http://www.genarts.com (2009), GenArts, Inc.

 

Video 

The Lord of the Rings: The Two Towers ‐ Extended Edition, Part Four: The Battle for Middle‐Earth Begins ‐ Visual Effects ‐ WETA Digital Feature, Newline Cinema (2002)

The Roots of The Matrix – The Hard Problem: The Science Behind The Fiction (special DVD feature of ‘The Ultimate Matrix Collection), Warner Bros. (2003)

Batman Returns, Warner Bros. (1992)

Big Ad, George Patterson Y&R for Carlton Draught (2005)

Billions, Fredrik Bond for Unilever’s Axe/Lynx (2006)