Genetic Programming as a Tool for novel Creation

CS 621 Seminar

Sri Raj Paul(08305034) Course InstructorBalamurali(08405401) Prof. Pushpak Bhattacharyya

THE WAY WE GO.

Invention & Patent

AI & Invention

Genetic Algorithm

Genetic Programming

GP Invention Machine

Conclusion

INVENTIONWhat is it? a new form, composition of matter, device, or processWhat is a patent?a set of exclusive rights granted by a state to an inventor or his assignee for a fixed period of time in exchange for a disclosure of an invention

IS EVERY INVENTION PATENTABLE?

Is an improvement over a patented invention

Result is equal to or better than a result that was placed .

Result is publishable in its own right as a new scientific result.

The result solves a problem of indisputable difficulty in its field.

AI & INVENTIONA new idea that can be logically deduced from facts that are known in a field, using transformations that are known in a field, is not considered to be inventive

Obtaining implication of given facts and rules -- Hallmark of intelligence ~ Prof. PB slides

Result: AI based on reasoning and logic cannot Invent !

GENETIC ALGORITHMInspired by evolutionary biologyA solution represented as a chromosomeMethodologyInitializationSelectionReproductionCrossover MutationTermination

GENETIC PROGRAMMINGGP applies the approach of the genetic algorithm to the space of possible computer programs Computer programs are the basic way for expressing the solutions to a wide variety of problemsGenetic programming now routinely delivers high-return human-competitive machine intelligenceHigh -> high AI (artificial-to-intelligence ) ratioRoutine -> repeating successfully on different set of problemshuman-competitive -> is patentable in a sense

GP OPERATORSReproductionCrossing overMutationArchitecture Alteration operation

GP FLOW CHART

Source: John Koza slides

PREPARATORY STEPSSource: www.genetic-programming.comThe human user communicates the high-level statement of the problem to the genetic programming using preparatory steps

FUNCTIONAL SET AND TERMINAL SETAlphabets of the programs to be made

The terminal set consists of the variables and constants of the programs

The functions are several mathematical functions and other more complex functions

FITNESS MEASURESpecifies what needs to be done

The primary mechanism for communicating the high-level statement of the problems requirements

The first two preparatory steps define the search space whereas the fitness measure implicitly specifies the searchs desired goal.

CONTROL PARAMETERS AND TERMINATIONThese steps are administrativeControl parameter:population size.probabilities of performing the genetic operationsthe maximum size for programsTermination criterionmaximum number of generationsmay manually monitor and manually terminateMethod of designating the resultsingle best-so-far individual

GP INVENTION MACHINEProblem : To create a low pass filter without patent infringement of Ladder filter.below 1,000 Hz Pass bandabove 2,000 Hz Stop Band

Ladder FilterSource: Genetic Programming as a Darwinian Invention Machine

PROGRAM ARCHITECTURETopology-modifying functionsalter the circuit topologyComponent-creating functionsinsert components into the circuitDevelopment-controlling functionscontrol the development processArithmetic-performing functionsspecify the numerical value of the componentAutomatically defined functionsenable certain substructures of the circuit to be reused

PREPARATORY STEPSInitial CircuitProgram ArchitectureFunctionsTerminalsFitnessControl ParametersTermination

INITIAL CIRCUITTest Fixture fixed substructureprovides access to the circuit's external inputpermits probing of the circuit's outputEmbryo development occurs in the embryoSource: Genetic Programming as a Darwinian Invention Machine

FUNCTIONS AND TERMINALSF= {C, L, SERIES, PARALLEL, FLIP, TVIA0, , TVIA7, NOOP}

Tccs = {END, CUT}ccs- construction continuing sub-treeEND makes the modifiable component with which it is associated non-modifiableCUT causes the component to be removed from the circuit

Taps = {R}aps- arithmetic-performing sub-tree

FITNESSMeasurement the circuits behavior in the frequency domain101 Signals from 1 Hz and 100,000 Hz divided using a logarithmic scale is givenError measured using Formula

Circuits similarity to the to-be-avoided ladder filtersub graph of the given circuit that is matching to a sub graph of a ladder filterBoth are multiplied to get over all fitnessSmaller the overall value of fitness is better

CONTROL PARAMETERS AND TERMINATIONControl ParametersPopulation size, M is 1,950,000Circuit constructing program tree size is 300

TerminationGoal is to generate a variety of 100%-compliant circuitsNumerous 100%-compliant circuits were harvestedManually terminated

RESULTSBased on Matching factor & frequency response more than 8 suitable offspring's were selected.

One of the result was elliptic filter(1927,Caur) Which is patented!Source: Genetic Programming as a Darwinian Invention Machine

CONCLUSIONGP can automatically create design thatsatisfies new specificationAvoids prior art

If a suitable fitness criteria can be found ,GP can be used in any field for invention

REFERENCEJ.R. Koza, F.H. Bennett III, and O. Stiffelman. 1999 Genetic Programming as a Darwinian Invention Machine. EuroGP99, LNCS 1598, pp. 93-108, Springer-Verlag Berlin Heidelberg 1999John R. Koza, Martin A. Keane, Matthew J. Streeter, "Routine High-Return Human-Competitive Evolvable Hardware," eh,pp.3, 2004 NASA/DoD Conference on Evolvable Hardware (EH'04), 2004http://www.genetic-programming.comhttp://en.wikipedia.org/wiki/Genetic_programmingProf. Pushpak Bhattacharyya slides

Thank You

Source: www.genetic-programming.comNext >>

Source: www.genetic-programming.comNext >>

Source: www.genetic-programming.comNext >>

***Invention is a highly creative process.Discovery consists of seeing what everybody has seen and thinking what nobody has thought. Albert Szent-Gyorgyi.An open curious mind enables one to see beyond what is known. Inventors think out-of-box .Hell, there are no rules here we're trying to accomplish something new. Thomas A Edision

An invention that is novel and not obvious to those who are skilled in the same field may be able to obtain the legal protection of a patent.

Is an improvement over a patented invention : eg plasma TV from normal CRT television

Result is equal to or better than a result that was placed : eg Creating insulin in less number of steps

Result is publishable in its own right as a new scientific result : eg finding of new species of rice variety .

The result solves a problem of indisputable difficulty in its field : eg finding whether a number is primes is in P

*Eg : A ->B and B->C A-> C is not an invention.

These two points contradicts : The second point is the basis of the most of the ai system, ie everything drives on a logic.

Interestingly, everyday usage parallels the law concerning the point that a lack of logic is a precondition for inventiveness: newton example of law of gravity:

of course, when we say that the invention process is inherently illogical, we do not mean that logical thinking is not helpful to inventors or that inventors are oblivious to logic. Logical thinking often plays the important role of setting the stage for an invention. Although logical thinking may play a role in invention and creativity, at the end of the day, the critical element is a logical discontinuity from established ideas.

Two of the basic process that takes mankind to new frontiers:Invention itselfEvolution

*

Genetic algorithms are a particular class of evolutionary algorithms (also known as evolutionary computation) that use techniques inspired by evolutionary biology such as inheritance, mutation, selection, and crossover (also called recombination).

*

The AI ratio (the artificial-to-intelligence ratio) of a problem-solving method as the ratio of that which is delivered by the automated operation of the artificial method to the amount of intelligence that is supplied by the human applying the method to a particular problem

A problem solving method is routine if it is general and relatively little human effort is required to get the method to successfully handle new problems within a particular domain and to successfully handle new problems from a different domain.

Result is Human competitive means :(A) The result was patented as an invention in the past, is an improvement over a patented invention, or would qualify today as a patentable new invention. (B) The result is equal to or better than a result that was accepted as a new scientific result at the time when it was published in a peer-reviewed scientific journal. (C) The result is equal to or better than a result that was placed into a database or archive of results maintained by an internationally recognized panel of scientific experts. (D) The result is publishable in its own right as a new scientific result independent of the fact that the result was mechanically created. (E) The result is equal to or better than the most recent human-created solution to a long-standing problem for which there has been a succession of increasingly better human-created solutions. (F) The result is equal to or better than a result that was considered an achievement in its field at the time it was first discovered. (G) The result solves a problem of indisputable difficulty in its field. (H) The result holds its own or wins a regulated competition involving human contestants (in the form of either live human players or human-written computer programs).

*Mutation:Pick point from 1 to NUMBER-OF-POINTSDelete subtree at the picked pointGrow new subtree at the mutation point in same way as generated trees for initial random population (generation 0)The result is a syntactically valid executable program

Corssing over:Randomly pick a number from 1 to NUMBER-OF-POINTS for 1st parentIndependently randomly pick a number for 2nd parentThe result is a syntactically valid executable programPut the offspring into the next generation of the populationIdentify the subtrees rooted at the two picked points

**

Genetic programming starts from a high-level statement of the requirements of a problem

The human user communicates the high-level statement of the problem to the genetic programming using preparatory steps

require the human user to specify*

For eg

Arithmetic formula : arithmetic functions of addition, subtraction, multiplication, and division as well as a conditional branching operator. The terminal set may consist of the programs external inputs (independent variables) and numerical constants.

if the goal is to create a program which does the task of a controller then the basic function sets would be functions that operates on time-domain signals, including integrators, differentiators, leads, lags, gains, adders, subtractors, and the like. The terminal set defines the external parameters (independent variables ) and numerical constants required for the program to function (output prog) , for eg in the above case it would be somethings like reference signal or transfer signal etc.

For a maze solver, the terminal set would contain three commands: forward, right and left. The function set consists of the functions of the program. In the maze example the function set would contain: If "dot" then do x else do y.

*For example, if the goal is to get genetic programming to automatically synthesize an amplifier, the fitness function is the mechanism for telling genetic programming to synthesize a circuit that amplifies an incoming signal (as opposed to, say, a circuit that suppresses the low frequencies of an incoming signal or a circuit that computes the square root of the incoming signal).

Gun Firing Program : The fitness function is the distance that the bullet is off from the target.

For a genetic program to set the time of a clock, the fitness function would simply be the amount of time that the clock is wrong.

*The termination criterion may include a maximum number of generations to be run as well as a problem-specific success predicate.

one may manually monitor and manually terminate the run when the values of fitness for numerous successive best-of-generation individuals appear to have reached a plateau.

The single best-so-far individual is then harvested and designated as the result of the run.

*1.Campell filter(ladder filter) history ~1917 created , A simple filter is a one-input, one-output circuit that receives a signal and passes the frequency components of the incoming signal that lie in a specified range (called the passband) while suppressing the frequency components that lie in all other frequency ranges (the stopband).The desired lowpass filter is to pass all frequencies below 1,000 Hertz (Hz) and suppress all frequencies above 2,000 Hz.

2. Avoiding campel filter template.

*CCP circuit constructing program trees

(1) topology-modifying functions that alter the circuit topology, (2) component-creating functions that insert components into the circuit, (3) development controlling functions that control the development process by which the embryo and its successors is changed into a fully developed circuit(4) arithmetic-performing functions that appear in subtrees as argument(s) to the component-creating functionsand specify the numerical value of the component(5) automatically defined functions that enable certain substructures of the circuit to be reused (with parameterization).

**The test fixture is a fixed substructure composed of nonmodifiable wires and nonmodifiable electrical components. The test fixture provides access to the circuit's external input(s) and permits probing of the circuit's output.

An embryo is embedded into the test fixture. All development occurs in the embryo.

*FunctionC and L functions are component-creating functions that insert an inductor or capacitor into a developing circuit and that assign a numerical value to the inserted component. The SERIES and PARALLEL functions modify the topology of the developing circuit by performing a series or parallel division.The FLIP function reverse the polarity of a component. The eight VIA functions provides direct connectivity between two points within the developing circuit via one of eight numbered layers on the imaginary printed circuit board or piece of silicon on which the circuit resides.The NOOP function is a development-controlling function.

TerminalTccs -construction-continuing subtreeThe development-controlling END function makes the modifiable wire or modifiable component with which it is associated non-modifiable (thereby ending a particular developmental path). The CUT function causes the highlighted component to be removed from the circuit.

Taps - arithmetic-performing subtree R is a real number

*The netlist created by Genetic Programming becomes the input a 217,000-line SPICE (Simulation Program with Integrated Circuit Emphasis) simulation program for doing frequency testing

d(fi) = difference in amplitude of target voltage and observed.W(d(fi) , fi ) = weight associated for the difference in voltage at a frequency.eg: At 50 Hz target output is 1V. The weights associated are 0 if d(50)=0 , 1 if d(50) < 30 mV, 10 if d(50) > 30 mV.

The similarity between circuits can be found using subgraph isomorphism algorithms.

**Had we conducted the experiment between 1917 and 1927 we could have got a patent for a new filter!

36 Human competetive results of which two are patentable.*Tell the example of A.R .Rehman Music.*******