1

Heuristic Programming in Business Solutions

ISECON’02

Kurt F. Lauckner

Department of Computer Science

Eastern Michigan University

csc_lauckner@online.emich.edu

2

What is “Soft Computing”

Often called Heuristic programmingIncludes:

Expert Systems and Fuzzy LogicNeural NetworksEvolutionary Systems

Genetic AlgorithmsGenetic ProgrammingOther Evolutionary Structures

3

Expert SystemsExpert systems are also called Rule-based systems.

Expert’s expertise is built into the program through a collection of rules.

The desired program functions at the same level as the human expert.

The rules are typically of the form:If (some condition) then (some action)Example: If (gas near empty AND going on long trip) then

(stop at gas station AND fill the gas tank AND check the oil).

EXCON: An expert system used by DEC

4

Expert Systems

Two major parts of an expert system:The knowledge base: The collection of rules that

make up the expert system.The inference engine: A program that uses the rules

by making several passes over them. On each pass, the inference engine looks for all rules whose

condition is satisfied (if part).It then takes the action (then part) and makes another pass over all

the rules looking for matching condition.This goes on until no rules’ conditions are matched.The results are all those action parts left.

5

Expert SystemsInference engines can pass through the rules in

different directions:Forward chaining: Going from a rule’s condition

to a rule’s action and using the action as a new condition.

Backward chaining: Goes in the other direction.Example: Medical doctors use both.

Forward chaining: Going to the doctor with symptoms (stomach pain). The doctor will come up with a diagnosis (ulcer).

Backward chaining: The doctor asks if patient has been eating green apples knowing green apples cause stomach aches.

6

Expert System -- Case StudyBusiness Resource Software, Inc. (BRS) has a

series of expert systems products that are a Knowledge Base of Business Experts. Their clientele include:

American ExpressAT&TBack-In-A-FlashBoeingCentury 21Dun & BradstreetE.I.DuPontEastman KodakErnst & YoungFederal ExpressHarvard UniversityIBM Corp.

J.C.PennyMcDonaldsMotorolaNew York LifeOlympusPfizerRoadway ExpressShell OilTexas InstrumentsToyotaWells Fargo BankXerox

7

BSR -- Expert System Products Plan Write Expert Edition --

Evaluates business idea and documents business plan with an Expert Assistant.

Plan Write for Marketing-- Produces a complete, fully documented marketing plan with defined objectives.

Plan Write for Hi-Tech Marketing -- A special version designed to assist the high technology industries to produce a complete, fully documented marketing plan with defined objectives and tactical lists for implementation.

Plan Write for Pricing -- Helps you to set the optimal price on any product or service.

Insight for Sales Strategy --Analyzes complex sales, identifies roadblocks, rates the potential for closing the sale, provides a pipeline of all forecasted sales.

Plan Write for Sales -- Develops and documents a plan for selling including strategy and tactics.

http://www.business-strategy-brs.com/customers.asp

8

Fuzzy Logic

Fuzzy logic comprises fuzzy sets and approximate reasoning

A fuzzy “fact” is any assertion or piece of information, and can have a “degree of truth”, usually a value between 0 and 1

Fuzziness: “A type of imprecision which is associated with ... classes in which there is no sharp transition from membership to non-membership” - Zadeh (1970)

9

Linguistic Variables

Linguistic variable: “a variable whose values are words or sentences in a natural or artificial language.” - Zadeh

Linguistic variables translate ordinary language into logical or numerical statements

Imprecision of linguistic variables makes them useful for reasoning

10

Linguistic VariablesIn fuzzy set theory we can translate the body temperatures of patients into linguistic values with an appropriate degree of membership µ.

With the linguistic variable fever, fuzzy membership functions can be defined: --low --normal --raised --strong_fever

1

0

µ(x)

x 96°F 98°F 100°F 102°F 104°F 106°F 108°F 110°F

strong_feverraisednormallow

11

Fuzzy Logic -- Case Study

Economic red flags in risk analysis and audit planning

Touche Ross [1974], Coopers and Lybrand [1977], and Price Waterhouse [1985] released a list of red flags for auditors

Deshmukh and Romine describe a fuzzy number based spreadsheet approach in assessing the risk of management fraud using red flags http://www.swcollege.com/acct/jac/jac12/jac12_article1.html

12

Fuzzy Logic -- Case Study

Uses FuziCalc® which provides a spreadsheet based environment that incorporates fuzzy numbers in the calculations

Define fuzzy sets: Strong internal control Weak internal control

Define that the internal control is effective if it detects or prevents 90% of the errors or irregularities

80% effective control may have a membership of 0.9 in a fuzzy set of "strong internal control" and a membership of 0.1 in the fuzzy set "weak internal control”

These concepts are useful in measuring and quantifying red flags

13

Fuzzy Logic -- Case Study

“The Loebbecke, Eining, and Willingham [1989] model stipulates that the probability of management fraud is a function of three factors: condition, attitude, and motivation.

“An effort must be made to aggregate red flags for each factor and then combine these three factors to determine the probability of management fraud.

“There are no statistical formulas to do this. “The fuzzy number based spreadsheet can help accomplish this

by mathematically manipulating fuzzy numbers. “If each red flag is considered equally important then the beliefs

for all red flags are equally weighed and the belief for the Condition factor can be calculated”

14

Neural Networks

Neuron: Basic building-block of the brain.There are several specialized types, but all have

the same basic structure:The basic structure of an animal neuron.

15

What is topology of NNs

NNs consist of processing elements (PEs) and weighted connections

16

What NNs are and why they are useful

Connection weights store the informationWeight values usually determined by learning

procedureEach PE acts independently of all othersEach PE relies only on local informationConnection pattern provides redundancy and

facilitates fault toleranceThe most common NN uses backpropagation

17

Processing Element

Propagation Function(Linear Combiner)

Activation Function

Xn

X3

X2

X1

Yj

Example:q = f(X,W)

Example (Sigmoid):

Wj 1

Wj 2

Wj 3

Wj n

Yj = f(X,W)

qj = xiwj ii=1

n

∑

qj

Yj =1

1+e−qj

18

Neural Network -- Case Study

Real Estate Example--Home price estimatorThe 17 inputs include:

Number of bedrooms, bathrooms, & other rooms.Garage type (attached, detached or none)Patio (yes or no)Air conditioningSpecialized windows (high R factor, security)Area code

One hidden layer with 8 nodes

19

Real Estate Case Study (Con’t)

Using 100 different homesWith 17 inputs and selling price of the homeUsed EasyNN a Windows under $40 neural

network program shell Trained network for several hoursResults of homes not known to the neural

network were estimated to within 2% of actual pricing

20

LVQ or Kohonen NetworksLVQ (Learning Vector Quantizer) neural

networks are often called Kohonen networksProbably second only to backpropagation in

number of applicationsNo rigorous mathematical derivation Seem to be more biologically oriented than

most paradigmsReduce dimensionality of inputsLVQ-I Makes use of unsupervised learning

Network learns from input patterns alone

21

Self-Organizing Feature Maps

Primary features of input cause corresponding local responses in the output PE field.

Are non-linear mappings of input space onto the output PE space (field).The mapping is referred to as a self-

organizing feature mapsExtensions of LVQ-I resemble the way

certain areas of the brain are organized

22

General Comments about LVQ-I

Main accomplishments of LVQ-I networks is they cluster input data patterns into classes, thus reducing dimensionality of data (e.g., automatically parameterizes the data)

Input vectors are usually normalizedWeight vector normalization is done for all

weights connected to a given output PE from all input PEs

23

Kohonen Neural Network -- Case StudyMarketing: A campaign is being run to increase sales of a product.  It is intended to do a targeted  mailshot to individuals most likely to be interested in the product.  There is information available from a consumer survey which includes information about current purchasers of the product. An unsupervised net is trained using information from the consumer survey.  The groups formed are examined and all individuals in the groups rich in existing purchasers are used as targets for the mailshot.

Data Cleaning:  Data being extracted from a database is being analysed but unfortunately the data is too inconsistent for the analysis technique.  An unsupervised net is used to create groups and the analysis is based on the individual groups instead of the entire database.http://www.neusciences.com/Technologies/nn_unsupervised.htm

24

Evolution--Darwin

“…if variations useful to any organic being do occur, assuredly individuals thus characterized will have the best chance of being preserved in the struggle for life; and from the strong principle of inheritance they will tend to produce offspring similarly characterized. This principle of preservation, I have called, for the sake of brevity, Natural Selection.”

--C. Darwin, 1859

25

Evolution--Darwin

Reproduction of individuals in the population Variation that affects the likelihood of

survival of individualsHeredity in reproduction (e.g.,like begets like) Competition caused by finite resources.

Translating Darwin’s statement, the four essential preconditions for the occurrence of evolution by natural selection are:

--Banzhaf

26

Evolutionary Algorithms

Populations of solutions Innovation operations (e.g., mutation) Conservation operations (e.g.,crossover) Quality differentials (e.g.,graded fitness

function, richness of differences) Selection (e.g., differences in population

are used to select fittest.

Basic Ingredients of EAs:

27

Genetic Algorithm--Holland Uses binary genome of fixed length

Crossover most common and usually of the is one-point crossover variety

Mutation used very sparingly Classifier systems used to select fittest consist of

three parts: Rule-based programming language Simulated market economy Genetic algorithm

1 0 0 1 1 1 0 1 0

1 0 0 1 1 1 0 1 0 1 0 0 1 1 1 1 1 1

1 1 0 1 0 1 1 1 1 1 1 0 1 0 1 0 1 0Parents Offspring

28

Genetic Algorithm -- Case Study

Improve model for tactical asset allocation

Improve model for international equity strategies

Improvements of up to 82% in cumulative portfolio value as compared with passive benchmark model

29

Evolutionary Programming

Stochastic decision making: GP uses pseudo-random numbers to mimic the randomness of natural evolution.

Program structures: GP assembles variable length program structures from basic units called functions and terminals.

Genetic operators: GP transforms the initial programs in the population using genetic operators (e.g. crossover, mutation, reproduction)

Simulated evolution of a population by means of fitness-based selection.

Essential Common Themes:

30

Primitives of Genetic Programs

Terminal set: terminals are the inputs to the GP program (named by fact they are terminals of a tree structure)

31

GP Primitives (Con’t)

Function set: these are the statements, operators, and functions available to the GP system.Boolean functions ( AND, NOT, OR, XOR etc.)Arithmetic functions (plus, minus, multiply etc.)Variable assignment functions ( x :=3)Indexed memory functionsConditional statements (IF, THEN, ELSE, CASE)Control transfer statements (GOTO, CALL, JMP)Loop statements (WHILE…DO, REPEAT…)Subroutines (for example a robot: turn left, stop…)

32

Executable Program StructuresFunctions and terminals must be assembled into

some structure that can be executed and evaluated.Two methods of initializing tree structure:

Full method of tree initializationPick functions as nodes until maximum depth reachedPick terminals (leaves) of the tree from terminal set

Grow method of tree initializationPick randomly from functions and terminal setsIf function selected continue with that branchIf terminal selected the branch ends

33

Execution of Structure

In the tree structure use the pre-order traversal to test fitness of all individuals of the population

Apply selection algorithmUse crossover, mutation and reproduction to

create new populationContinue until termination criterion met or

maximum number of generations reached

34

Genetic Programming -- Case Study

GP paradigm tool STROGANOFF (1990s)Used in predicting USD/SF currency exchange

ratesUsed in predicting movements of the Dow Jones

industrial indexUsed in predicting movements of the Nikkei225

index of the Tokyo Stock ExchangeOther applications: derivative markets, banking

and finance, marketing